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Study in Brief 


This report presents results from the CPB-PBS Ready To Learn PEG+CAT Content Study. PEG+CAT is a unique 
transmedia property that emphasizes early mathematics and problem solving. Each episode is structured around 
a specific mathematical problem the characters are trying to solve together. It includes video episodes, interstitial 
video, online games, and a tablet-based app, that allows children and their families to engage with the same 
characters, settings, and narratives on multiple devices, across various physical and social settings. The study 
examined how a sample of 59 four-year-old children engaged with and learned from selected sets of PEG+CAT 
videos and interactive games over five less-than-hour-long interactions. These five independent, unmediated 
interactions took place in a controlled learning laboratory setting. 


This study is one of an interrelated series of studies included in the Ready To Learn summative evaluation, being 
conducted by EDC’s Center for Children and Technology and SRI Education’s Center for Technology in Learning 
(EDC/SRI). EDC/SRI are serving as the external summative evaluators on behalf of the Ready To Learn partnership 
among the US Department of Education, the Corporation for Public Broadcasting, and PBS. 


PEG+CAT Content Study 


Key Findings 


Children’s Learning: 


e Children’s performance improved significantly from pretest to post-test on one shape identification item (identifying a 
cylinder) on a researcher-developed measure aligned to the PEG+CAT study experience. 


e For 3 items (related to identifying 3-D shapes), there were non-significant gains in performance between pre- and 
post-test. 


e Children’s performance on the standardized assessment (REMA) improved modestly from pretest to posttest. 
The result is positive (statistically significant) but not conclusive. 


Pere Wier e hs Weenies: 


e Nearly all parents reported strong, positive impressions of PEG+CAT and viewed the resources as having 
considerable potential to support children’s mathematical learning. A number of parents reported that interacting 
with the PEG+CAT materials appeared to influence children’s behavior at home and that children talked about 
PEG+CAT at home, after and in-between study sessions. 


e Half of the participating parents reported that they worked with their children on activities related to PEG+CAT 
at home. 


Children’s Engagement: 


e Children showed signs of positive engagement, like watching intently, counting along, or “interacting” with 
characters while watching PEG+CAT videos and playing PEG+CAT games. 


¢ Children consistently sat still and paid focused attention during video viewing sessions. Their level of engagement 
with interactive games varied. 


e Most children were able to identify and talk about the characters, setting, and other story elements of the videos, 
but fewer were able to describe the mathematical problem and the solution around which the episode revolved. 


e Children were able to engage with the games independently during the majority of sessions, but some form of 
support was necessary during other sessions. 
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Introduction 


EDC and SRI researchers conducted this study to provide new information about how preschool children may 
learn as a result of direct engagement with transmedia resources outside of formal instructional environments like 
preschool classrooms. These transmedia experiences, developed by public media producers and made widely 
available by PBS, included use of PEG+CAT video episodes, interstitial video, online games, and a tablet-based 
app. These resources allow children and their families to engage with the same characters, settings, and narratives 
on multiple devices, across various physical and social settings. 


This study contributes to several important areas of research. First, this study focuses on how new educational 
media resources may support learning for children growing up in low-income communities where there are often 
fewer resources available to children that can help them to develop the skills and content knowledge associated 
with success in school. Second, by focusing on PEG+CAT, created by Jennifer Oxley and Billy Aronson, and 
produced by the Fred Rogers Company, the study extends a long line of research that investigates the potential 
benefits of public media developed materials. 


Public Media, Transmedia, and PEG+CAT 


The Ready To Learn program and the public media organizations that produce educational content with support from 
Ready To Learn are focused on producing and deploying educational content to support learning in homes, at school, 
and in out of school settings. The current initiative is focused on producing high-quality early learning materials that 
support children’s exploration and development at home and outside of traditional instructional environments. 


Multiple interventions involving adult-supported engagement with public media videos and transmedia suites 
have been effective at supporting math, literacy, and science learning in lab, school, and home settings, and have 
been associated with gains in literacy, science, and math, as well as school readiness skills (e.g., Fisch, 2004, 
Penuel et al, 2012, Pasnik & Llorente, 2013). Such resources often provide access to dynamic, multi-sensory 
representations of concepts, places, or people that would not be practical (or sometimes, possible) otherwise. 
Additionally, a number of public media interventions that target young children—and the digital resources on 
which they are based—attend carefully to pedagogical design, and the learning and developmental needs of the 
participants (e.g., Fisch, 2004, Piotrowski & Linebarger, 2010). 
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The Corporation for Public Broadcasting and PBS, in partnership with the U.S. Department of Education and 
their producing partners, are pursuing a “transmedia approach” to supporting early mathematics. PEG+CAT is 
a centerpiece of this effort. The PEG+CAT experience includes video episodes, interstitial video, online games, 
and a tablet-based app that allows children and their families to engage with the same characters, settings, and 
narratives on multiple devices, across various physical and social settings. The title characters in the program, 
Peg and Cat, along with their friends and adversaries, present children with an animated, fictional universe where 
solving important problems requires children to explore and rehearse both mathematics concepts and social- 
emotional skills. 


Developers of transmedia properties intend to create experiences that can support both intentional and incidental 
learning opportunities, across a range of settings. Children can view videos or play games at home, perhaps while 
waiting for care providers to attend to their own essential tasks. Children with access to the Internet via portable 
digital devices can play an app or video while in transit to school or daycare sites. And children can engage in these 
activities on their own, with siblings, with friends, and with caregivers. Each of these experiences can potentially 
contribute to children’s learning and understanding of targeted skills, concepts and language. 


PEG+CAT producers identified two principal goals for this transmedia property: 1) to enhance children’s learning 
of mathematics (skills and content), and 2) to support children in becoming persistent and resourceful in solving 
problems. This study focuses on the first of these goals. 


Early Mathematics 


The study focuses specifically on how transmedia experiences can support young children’s learning about shapes 
and patterns outside the preschool classroom. Parents, educators, and researchers all recognize that the years before 
children begin kindergarten are essential for developing skills and knowledge foundations that will be crucial for learning 
in later years (Chetty, Friedman, Hilger, Saez, Schazenbach, & Yagan, 2011; Isaacs, 2008). There is growing recognition, 
too, about the importance of early mathematics learning, and increasing awareness of the tremendous potential all 
children have to develop a broad range of quantitative thinking skills. Early mathematics achievement is a strong predictor 
of later school achievement, and this predictive power is, arguably, greater than the predictive power of early literacy 
achievement (Claessens, Duncan, & Engel, 2009; Duncan et al., 2007). 


The National Association for the Education of Young Children (NAEYC) and the National Council of Teachers of Mathematics 
(NCTM) jointly have called attention to the need for appropriate, challenging, and effective early childhood mathematics 
programs (2002; 2010). Yet most preschool teachers are not trained in early mathematics content, the developmental 
trajectory of young children’s acquisition of mathematics skills, or teaching strategies to promote mathematics learning 
(Ginsburg, Lee, & Boyd, 2008). Many existing methods of teaching early mathematics appear insufficient to help many 
children, especially those from lower income households and English learners, achieve mathematical learning outcomes 
equivalent to their English-speaking middle-class peers. Children who fall behind in mathematics early on face long odds 
of ever catching up to their more mathematically proficient peers when it comes to high-school graduation rates, college 
readiness, and income as adults (Duncan, et. al., 2007; NAEYC & NCTM, 2010). 


PEG+CAT Content Study 


A substantial body of research shows that high-quality mathematics instruction can lead to improved mathematics 
outcomes for disadvantaged children, and that teachers and caregivers are able to support children’s early 
mathematics learning when provided with appropriate guidance and information (Casey et al., 2008; Kersch, 
Casey, & Mercer Young, 2008; Wolfgang, Stannard, & Jones, 2001). Preschool classroom curricula and curricular 
supplements have been shown to support learning gains for children when they are well aligned with children’s 
physical, social, and cognitive development (Starkey, Klein, & Wakeley, 2004; Chambers et al., 2006; Clements & 
Sarama, 2008; Linebarger & Piotrowski, 2009; Neuman, Newman, & Dwyer, 2010; Penuel et al, 2011). 


Early Mathematics Outside Preschool Environments 


Local efforts to improve formal early learning environments can include adopting new curricula, using new resources, 
expanding teacher preparation and offering professional development. While these in-classroom investments are 
important, there is also evidence that supporting children in learning early mathematics through structured, supported 
engagements between children and their parents and caregivers is a promising strategy (Starkey, Klein, & Wakeley, 
2004, McCarthy et al, 2012). Home environments and family relationships are important contexts for mathematics-rich 
interactions for preschool and school-aged children (Anderson, 1997; Ginsburg, Inoue, & Seo, 1999; Swarchuk, 2009; 
Goldman & Booker, 2009; Goldman et al., 2010; Esmonde et al., 2011). Activities that take place at home, and among 
parents/caregivers and children can influence preschoolers’ development of mathematics skills and knowledge (Young- 
Loveridge, 1989; LeFevre, Clarke, & Stringer, 2002; Aubrey, Bottle, & Godfrey, 2003; Sheldon & Epstein, 2005; LeFevre 
et al., 2009; Anders et al., 2012) There is little research on the influence of children’s independent experiences with 
educational content and how such engagements may support mathematics learning. This study is designed to make a 
modest contribution to filling this knowledge gap. 
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Study Design 


The purpose of the PEG+CAT Content Study was to explore, in a controlled environment, to what extent children 
can learn mathematics from PEG+CAT content outside of instructional environments and relationships, how 
parents' perceived the resources, and how well children are able to engage with these resources independently. 
This study was conducted in the spring of 2014 in the New York City and San Francisco Bay Areas. 


All children who participated in the study were provided with opportunities to view PEG+CAT videos and play 
PEG+CAT digital games and activities under controlled, non-instructional conditions, in “lab” spaces at EDC and 
SRI offices. The study used a pre/post design with no comparison or control group. 


Researchers collected data using standard and researcher-developed, experience aligned assessments, researcher 
observations, and parent observations of children’s behaviors during and outside study times. 


Research Questions 


The PEG+CAT Content Study design focused on three research questions. The first question addressed the degree 
to which children learned new mathematics knowledge and skills. Two assessments, one a standard measure (the 
Research Based Early Mathematics Assessment (REMA), short form), and one developed by the research team 
(PEG+CAT Item Sets, PCIS), were used to collect data to address this question: 


RQ1: Do children who engage with selected PEG+CAT videos and games independently in a controlled, non- 
instructional environment over the course of five sessions improve in target mathematics skills as measured by: 


(a) a standardized early childhood mathematics assessment; and 


(b) a researcher-developed measure, aligned to the videos and games included in the study experience? 


The second question addressed parents’ perspectives of the transmedia content and their children’s mathematics 
experiences. Parents were asked to respond to a set of open-ended questions during the fifth week of the study 
that asked about their impression of the resources as supportive of learning, and about their child’s behavior with 
regard to any references to the PEG+CAT resources. 


1 “Parents” refers to all primary caregivers, no matter their relationship to the child. 
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RQ2: How do parents view the potential of PEG+CAT content to support children’s mathematics learning? 


The third question addressed children’s engagement with the transmedia and also addressed the potential 
need for mediation or support for children’s use of the games, videos and activities. Researchers completed 
observations during children’s study activities, and parents also completed observation check lists to contribute 
their perspective on their children’s engagement and perceived need for any assistance. 


RQ3: How do children attend to particular PEG+CAT media experiences? What assistance or support, if any, do 
they require to participate while engaging in the media experiences? 


PEG+CAT Content Study 


Sample 


The final study sample included 59 children (aged between 4 years, 1 month and 5 years, 5 months) from low-income 
communities in the New York metropolitan and San Francisco Bay areas. The initial study sample included two 
additional children, one of whom withdrew from the study after the first session (CA) and the other who withdrew 
after the second session (NY). The research team recruited children through existing relationships with Head Start 
centers, public preschools, and community based programs where 50% or more of the children served are from 
low-income backgrounds (i.e., qualify for subsidized preschool and/or free or reduced price lunch). The recruitment 
process involved: identifying and talking with preschool and district program directors about the study; distributing 
informational flyers to parents at preschool sites; talking with teachers and other care providers about the study; and 
talking with parents at center-related events about the study. 


Families attended a total of five sessions during the study. Participating families received a stipend of $50 per visit 
to EDC/SRI in acknowledgement of their participation. Where appropriate, families also received stipends to cover 
travel costs to the study site. 


Researchers asked parents/caregivers to complete surveys that would provide background information about the 
children (and families) participating in the study, including basic demographic information and family perspectives on 
mathematics and technology. Parent surveys indicate that the study sample was ethnically diverse: 60% Hispanic/ 
Latino, 25% African American, and 14% Mixed/Other. In terms of technology at home, a majority of NY families 
had broadband Internet access (67%), but far fewer CA families reported such access (23%). Television was the 
most common media/technology platform used by families, particularly for supporting learning. According to parent 
reports, 40% of participating children watched educational TV daily, compared to 20% who played digital games. 


Prior to participating in the study, parents tended to agree or strongly agree that young children can learn math, 
that math learning can happen everywhere, and that parents can support math learning. Most parents reported 
supporting math learning at home at very high rates, especially for well understood mathematics such as counting 
(97%), recognizing or drawing shapes (92%), number identification (85%), recognizing/making patterns (75%), 
addition (73%), and subtraction (61%). 


Report to the Ready To Learn Initiative 7 


PEG+CAT Content Study 


Treatment 


Researchers conducted a detailed review of all learning materials developed by PEG+CAT producers, closely 
attending to focal mathematics skills and the strategies designers built into the resources to support learning. 
Following review, researchers selected specific PEG+CAT content that targeted two mathematics skills: patterns 
and shapes (3D/2D). Researchers selected and sequenced the videos and games to create an experience that 
would allow children to have more than one opportunity to engage with those particular mathematics skills. Below 
we provide details about the resources and the sequence in which children experienced them. 


Focal Skills and Media Resources 


Researchers identified and chose to include the strongest assets focused on patterns and shapes. The patterns games and 
episodes selected focused on both auditory patterns (for example, Beethoven playing the first notes to his 5th Symphony) 
and visual patterns (for example, the diagrams of short-short-short-long sausages). The patterns-focused resources included 
The Beethoven Problem, Big Gig’ Patterns game, The Dinosaur Problem and the Chicken Dance game. 


The videos and games that focused on 2D and 3D shapes (e.g., spheres, cylinders, and pyramids, which are less 
common in preschool) included The Golden Pyramid Problem, The Sparkling Sphere Problem, Magical Shape 
Hunt game, and Big Gig Magic Cylinder song. 


Researchers also elected to include resources that focused on counting and number relationships because these 
are skills with which children were likely to be familiar coming into the study and provided a way to introduce the 
characters, narrative, and media experiences they would encounter in subsequent sessions. The introduction video 
(The Three Bears Problem), helped introduce participants to the characters and format of PEG+CAT. Additionally, 
The Dinosaur Problem, which was included in the study experience for its focus on patterns, also contains content 
related to number relationships and counting. 


Technology 


The transmedia content included in the study dictated the selection of the technology platforms. Videos and 
games were accessed using a laptop (Chromebook), while activities included in the Big Gig App required use of 
a tablet. We chose to use the Nabi, a touch-screen tablet specifically designed for use by younger children, that 


2 A newer version of the Big Gig app was released after the study concluded. 
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includes such features as bright and engaging colors and a soft, easy-to-hold, protective wrapper that minimizes 
potential damage to the screen if the device is dropped. Researchers constructed a website that provided easy 
access to the study videos and online games (see figure 1 below). 


Figure 1. Ready To Learn Content Study Web Site 


RTL Content Study Transmedia Links 


Session Video Episode Interactive Game 
1 The Three Bears (Episode 107A) Big Gig App (Count by Ones) 
2 The Dinosaur Problem (Episode 108A) Chicken Dance game 
3 The Beethoven Problem (Episode 108B) Big Gig App (Patterns) 
4 The Sparkling Sphere Problem (Episiode 112B) Big Gig App (Magic Cylinder) 
5 The Golden Pyramid Problem (Episode 102B) Magical Shape Hunt game 


Typical Session Experience 


Parents who agreed to join the study were asked to attend five sessions over a five-week period. Each session 
included a specific set of activities that would take between 45 and 75 minutes, including a break and small snack. 
Parents were asked to attend each of the sessions with their child. Children who missed two or more sessions 
were excluded from the study, and the majority of participants attended all five sessions. 


The principal learning opportunity during each session was children’s direct engagement with PEG+CAT resources. 
Beyond the careful curation of resources, there was no formal instruction, feedback, or guidance provided to the 
children during study sessions from researchers or parents. Researchers were present and provided assistance 
as requested by children, making sure the child had the best opportunity to participate in each session. This 
included ensuring the technology was functioning properly (i.e., correct video was playing, correct game was 
loaded), answering any questions the child may have had about how to start playing a game, and making sure the 
child was comfortable. During a typical session, a researcher would work with the same family from week to week. 
Sessions began with assessment tasks, followed by video viewing and/or game play. During these activities, 
parents and researchers would observe, make notes, and complete checklists while the child interacted with the 
video and game resources. At the end of each viewing/play session, researchers would ask the child a short list of 
questions about the experience. Each session concluded with a round of assessment tasks. Parents were asked 
to observe, but not interact with their children during sessions. 
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Below is a detailed description of activities that took place during each of the five sessions of the study: 


Session activities 


Session 1: 


e Parents and children participated in a short orientation to the study, the lab setting, and the materials children 
would view and with which they would be engage. 


e Children participated in a pre-assessment with a trained researcher using a standardized assessment. They also watched 
one video and played one game. The assessment took place in a room where the child’s parent was present, though was 
engaged in the completion of a parent survey. The session lasted approximately 45-55 minutes. 


e Snacks were provided for children and parents. Figure 2. The Three Bears Problem 


e Selected content: The Three Bears Problem (Episode 104A) (See 
figure 2), Big Gig App - counting by 1s 


Sessions 2 - 4: 


e Children a) completed a brief set of experience-aligned pre- 
assessment tasks, (b) engaged in a media experience (1 video and 
1 online game/activity); (c) completed a small set of experience- 
aligned post assessment tasks. 


e Parents were present during the media experience and assessments but did not participate in the media or 
assessment activities. Instead, parents observed their children’s media experiences and completed a short 
reflection form noting their child’s interactions with the media, and documented their reflections and perspectives 
on a range of issues such as: their child’s level of engagement, the kinds of mathematics present in the media, 
their child’s ability to navigate a game or activity, the nonacademic content present in an episode or game, and 
how the content might relate to other activities the child typically does at home. Sessions lasted approximately 
35-45 minutes. 


e Snacks were provided for children and parents. Figure 3. The Dinosaur Problem 


e Selected content: 
Session 2: The Beethoven Problem (Episode 108B), Big Gig 
Patterns game 
Session 3: The Dinosaur Problem (Episode 108A) (see figure 3), 
Chicken Dance game 
Session 4: The Golden Pyramid Problem (Episode 102B), 
Magical Shape Hunt game 


a 

é 

- 
PoC 
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Session 5: 


e Children were pre-tested with a small set of experience-aligned 
tasks. Following a final media experience (1 video and 1 online 
game/activity), children were post-tested using a small set of 
media-aligned items and the standardized assessment. This 


session lasted approximately 60-75 minutes. 


e Snacks were provided for children and parents during a 
15-minute break. 


e Selected content: The Sparking Sphere Problem (Episode 112B) 
(see figure 4), Big Gig Magic Cylinder (see figure 5) song game 


Figure 4. The Sparking 


Table 1. Sequence of PEG+CAT Content Study Transmedia 


D- t= Im Oe) | (=Leh wd (0) | 


Session 1 


Session 2 


Session 3 


Session 4 


Session 5 


Counting; 
Number 
relationships 


Patterns 


Patterns 


Shapes 


Shapes 
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REMA pretest 
Parent Survey 
Observation 
Parent checklist 


Session 2+3 pretest 
Observation 
Parent checklist 
Session 2 posttest 


Observation 
Parent checklist 
Session 3 posttest 


Session 4+5 pretest 
Observation 
Parent checklist 
Session 4 posttest 


Observation 
Parent checklist 
Session 5 posttest 
REMA posttest 


Three Bears Problem 
(Chromebook) 


The Dinosaur 
Problem 
(Chromebook) 


The Beethoven 
Problem 
(Chromebook) 


The Sparkling Sphere 
(Chromebook) 


The Golden 
Pyramid Problem 
(Chromebook) 


The Big Gig Count by Ones 
(Nabi Tablet) 


Chicken Dance 
(Chromebook) 


The Big Gig Patterns 
(Nabi Tablet) 


The Big Gig Magic Cylinder 
Song 
(Nabi Tablet) 


The Magical Shape Hunt 
(Chromebook) 


Methods 


To address the research questions, researchers relied on a variety of data sources, including direct assessments 
of children’s mathematical learning; researchers’ observations of children interacting with the PEG+CAT videos 
and games; and parent surveys and written observations/reflections. Table 2 highlights the data sources related 
to specific research questions and Table 3 shows the data sources and the number of responses for each. 


Table 2. Research Questions and Methods 


PEG+CAT Item Sets Parent 
(Researcher- Researchers’ Parent 


‘ checklist and 
Co (=\V(=}[o) of ={e| observations : Survey 
reflection tool 
measure) 


RQ1: Children’s 
mathematical learning ff 


outcomes 


REMA 


Research questions (Standardized 
assessment) 


RQ2: Parents’ 
perspectives on Yi of 
educational potential 


of PEG+CAT materials 


RQ3: Children’s 
engagement with i wa 


PEG+CAT media 
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Table 3. Number of Participants for each Data Source 


REMA Assessment PCIS Assessment Parent 


Researcher checklist & Parent 

Pretest Posttest Pretest Posttest Observations -fectiontoo! Survey 
Session 1 60 NA NA NA 60 58 60 
Session 2 NA NA 60 60 60 60 NA 
Session 3 NA NA NA 59 59 a7 NA 
Session 4 NA NA 5g 58 59 58 NA 
Session 5 NA oo NA SE) SE) 60 NA 


Children were occasionally accompanied by more than one adult. 


Child Learning Outcomes 


To examine children’s mathematics knowledge, the research team relied on two tools. One was a standardized 
assessment of children’s mathematical learning, the Research-Based Early Mathematics Assessment (REMA), 
short form; Weiland et al., 2012) the best available standardized assessment of children’s mathematics skills. 


However, researchers needed an additional assessment because the PEG+CAT resources curated for the Ready To 
Learn content study experience addressed mathematical skills that are not necessarily the focus of the REMA €.g., 
auditory patterns and 3-D shapes). Researchers were also interested in assessing children’s understanding of focal 
mathematical skills immediately following the opportunity to learn them through exposure to the PEG+CAT resources, 
that is, during the sessions in which children had viewed videos and played the games that addressed the relevant skills. 
Therefore, researchers developed the PEG+CAT Item Sets (PCIS), which were closely aligned to PEG+CAT experience 
and administered before and after the sessions during which children engaged with media that targeted specific skills. 


Research-Based Early Mathematics Assessment (REMA) 


The REMA is designed to assess children’s mathematics learning in prekindergarten through second grade. The 
short version of the REMA, which was used in our study, was developed by Weiland and colleagues (2012) based 
on the full REMA (Clements, Sarama, & Lui, 2008); this assessment measures preschool and kindergarten children’s 
early numeracy and geometry skills. The 19 items selected to be part of the short version of the REMA assess 
mathematics skills that are considered essential in preschool and kindergarten (NGA/CCSSO, 2010; Clements & 
Sarama, 2009)—recognition of number and subitizing, shape composition, and patterning (Weiland et al., 2012). 


Each item includes a game-like activity that involves the assessor reading a verbal prompt and, at times, demonstrating 
with manipulatives. Children are required to provide a verbal response, point, or engage with manipulatives. 


Rasch analysis conducted on the short version of the REMA provides evidence of high item reliability (approximately 
1.00) and high person reliability (ranging from 0.68 to 0.76) (Weiland et al., 2012). Findings also indicate that the REMA 
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short form has adequate concurrent validity (correlations of 0.74 were reported with the full version of the REMA and the 
Woodcock Johnson Applied Problem subtest) and adequate discriminant validity (correlations of 0.64 were reported 
with the Peabody Picture Vocabulary Test 3rd Edition and the Letter Word DIF in the short version of the REMA). 


The PEG+CAT Item Sets (PCIS) 


To examine children’s understanding of the concepts and activities included in the PEG+CAT sessions, the research 
team developed a series of item sets specific to each session. Using information gathered from the videos and games, 
the PCIS were used to assess children’s understanding of shapes and patterns in the specific context of the session. 
Items in the PCIS involved game-like activities that require assessors to read a verbal prompt; in response, children 
were required to provide a verbal response, point, or engage with manipulatives. The items included in the PCIS are 
aligned to the concepts and videos shown in the PEG+CAT sessions (see Appendix A for the full assessment). 


Each item set was developed to examine the child’s learning in that particular session, and each item was designed 
to function descriptively within the set, focusing on particular skills present in the PEG+CAT games and videos 
(see above). The item sets were not designed to function as a battery or total score, and we did not analyze them 
as such. Instead, we analyzed each item individually from pre test to post test. 


Data collection of child learning measures 


Trained assessors administered the REMA before Session 1 and at the conclusion of Session 5, and the PCIS 
during sessions 2 through 5. 


The videos and games included in the content study experience were curated to provide two opportunities for 
children to learn about patterns and shapes, the two focal mathematical skills. Sessions 2 and 3 focused on patterns; 
sessions 4 and 5 focused on shapes. The PCIS pre-test for patterns was conducted at the start of session 2, and the 
post-test, which was broken into two segments and was conducted after sessions 2 and 3. Similarly, the PCIS pre- 
test for shapes was conducted at the start of session 4, and the post-tests were conducted at the end of sessions 
4 and 5 respectively. Because the pre-tests included topics related to two sessions, they were longer than the post- 
tests; also sessions 2 and 4, which included the pre-test as well as post-test, were approximately 15 minutes longer 
than sessions 3 and 5. Table 4 below provides a summary of the assessment schedule. 


Table 4. Ready To Learn Content Study Assessment Schedule 


Pretest Posttest 


Session 1 REMA pretest 

Session 2 PCIS Session 2 & 3 Pretest PCIS Session 2 posttest 

Session 3 one PCIS Session 3 posttest 

Session 4 PCIS Session 4 & 5 Pretest PCIS Session 4 posttest 

Session 5 one PCIS Session 5 posttest; REMA posttest 
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Data analysis for child learning measures 


To examine children’s understanding of the concepts and activities included in the PEG+CAT sessions, the team 
conducted a series of item-level descriptive analyses and a series of non-parametric analyses. The item sets 
were developed to be descriptive in nature at the item level, not as a battery assessment. Researchers conducted 
comparative analyses at the item level using the Wilcoxon matched-pairs signed-ranks test, which tests the 
equality of matched pairs of observations (Wilcoxon, 1945). The null hypothesis is that both distributions are the 
same. To correct for multiple comparisons, the Benjimini-Hochberg adjustment was used to control the expected 
proportion of incorrectly rejected null hypotheses. A paired t-test was used to analyze the REMA data. 


Parent's Views of PEG+CAT Transmedia Learning Materials 


Parent survey 


Parents of all participating children also completed a survey during Session 1 (See Appendix D). The purpose of the 
survey was to collect important background information (e.g., home language, parents’ educational background) 
about the families participating in the study and to gather information on how, if at all, families engage with 
video and digital games at home. The survey also asked respondents to provide information about home-based 
practices and activities that target the development of mathematical skills and understandings. 


Data analysis of parent survey data 


To examine the quantitative data collected in the survey, the research team conducted descriptive analysis by 
calculating frequencies and percentages for the binomial, categorical, and ordinal data, and means and standard 
deviations for the continuous data. To identify patterns in the qualitative responses, researchers examined all 
open-ended questions to develop potential coding categories. To identify patterns in the qualitative responses, 
researchers examined all open-ended questions to develop potential coding categories. Open-ended responses 
were then coded to identify meaningful patterns that could inform findings. 


Parent/Caregiver checklist and reflection tool 


Researchers asked parents/caregivers to observe all video viewing and game play sessions in which their child 
participated, and document their reflections in the Parent Checklist and Reflection Tool. The purpose of parents’ 
observations was to gather information about how parents viewed the PEG+CAT resources as well as their reflections 
on how their child was engaging with the materials, and what she or he might be learning as a result of the experience. 
The checklist included a series of yes-no questions, designed to capture parents’ impressions during the video 
viewing and the game play components of each session, and a set of open-ended questions that asked parents 
to articulate, if they chose to, more detailed observations about the PEG+CAT resources. During the final session, 
researchers included additional items that asked parents to reflect back on the experience as a whole, including any 
changes in child behavior at home that they felt were related to their participation in the study. 
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Data analysis of parent checklist and reflection tool 


Researchers conducted descriptive analysis of parents’ responses to the yes-no questions included the Parent 
Checklist and Reflection Tool. They also calculated the frequencies and percentages for each item, and determined 
whether patterns in parents’ responses were statistically significantly different across items. Researchers reviewed 
parents’ responses to the open-ended questions, and identified salient themes based on these reviews. Individual 
reflections were subsequently coded for the themes and themes were then counted and, in many cases, turned into 
percentages. Themes were then used to identify meaningful patterns across questions and respondents as well as 
to interpret the quantitative results. 


Children’s Engagement with PEG+CAT Transmedia 
Learning Materials 


Researcher observations 


To identify and describe children’s engagement patterns with PEG+CAT videos and interactive games included 
in the experience, researchers relied on close observations of children. The observations, which were guided by 
a structured observation protocol, were conducted during all sessions of the study while children were viewing 
videos and playing games. 


Researchers documented various features of children’s attention to the transmedia, and provided descriptions to 
support their observations; specifically, we were interested in identifying the extent to which children’s behaviors 
provided evidence of their engagement with the transmedia, whether they are able to complete the experience 
independently, and the frequency and type of assistance children required in order to complete the experience. 


At the conclusion of the video-viewing, researchers asked children to describe what they had noticed in the video, 
the problem Peg and Cat had encountered, and the solution that the characters had devised to address the 
problem. At the conclusion of game and app play, researchers asked children to describe what they had noticed 
in the game, and their opinion of the game, that is, whether children liked or disliked the game, why, and whether 
they had a part they liked best. 


Data analysis of observations 


Researchers analyzed observation data with the overarching goal of describing (a) patterns of attention observed 
among children during media experiences, and (b) the extent to which children required/requested adult support to 
engage in the viewing and/or online game/activity. These analyses offer a systematic description of the enactment 
of the experience—that is, how children engaged with the media during the study and what elements they were 
able to describe. 


Researchers obtained overall frequencies of the close-ended responses relating to whether or not a child exhibited 
certain behaviors during video viewing and gameplay (e.g., Child was sitting still, yes=1, no=0). Next, we conducted 
Pearson’s Chi-Square analyses to determine if there were differences across the five videos and five games in how 
children engaged with the specific digital assets. Due to the small sample size of 58-61 children or “cases” (the 
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sample size varied depending on the session; and one case included all the sessions one child completed), results 
of the chi-square analyses should be interpreted with caution, as there were several analyses that had cell sizes 
less than 5. 


Researchers used a grounded theory approach with constant comparison to analyze the open-ended responses. 
We compared the responses within a case and across sessions to identify the major themes. Four members of the 
research team individually coded 4 different cases (8% each for a total of 32% of the data), taking detailed notes 
on emerging themes. After discussing our individual findings, we developed a preliminary coding schema, which 
we then used to code an additional 5 cases (6.6% each for a total of 26.4% of the data). After meeting again to 
discuss our findings, we refined the coding schema (see Appendix H) and coded 6-7 additional cases each to 
complete coding of all observation data. 
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Results 


Researchers observed changes in children’s mathematical skills over the course of the five-week experience; 
parents views of PEG+CAT materials, including their potential to support children’s mathematical learning; and 
children’s engagement with the PEG+CAT experience and the extent to which they required adult support while 
interacting with the videos and games. 


Child Mathematics Outcomes 


Researchers analysis of the assessments (both the standardized measure, the REMA, and the researcher- 
developed measure, the PCIS) reveal the impact of PEG+CAT resources on children’s mathematical skills and the 
extent to which these resources represent opportunities to learn specific mathematical skills and concepts. 


Child learning outcomes for the Research-based Early Mathematics 
Assessment (REMA short form) 


Children’s performance on the standardized assessment (REMA) improved from 
pretest to posttest. While the result is positive, it is not conclusive. 


Researchers found a small, statistically significant increase (on average of 1.42 points) between pretest (mean = 
48.69, SD = 6.53) and posttest scores (mean = 50.12, SD= 5.38) when analyzing children’s performance on the 
standardized assessment. (Table 5, below, provides the complete results.) 


Table 5. Ready to Learn Content Study REMA Results 


Number of Standard 


WETar-1) (= Observations Mean Dediakon Min ED Significance 
Pretest 59 48.69 6.53 35 68 
Posttest Do 50.12 5.38 35 60 
Difference a9 1.42 4.90 -13 ue p <0.05 
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The significant difference in pre/post assessment scores should be interpreted conservatively. For context, children 
in the control condition of the 2013 PBS KIDS Transmedia RCT experienced more than twice the increase in their 
REMA scores from pretest to posttest (RCT Control: 2.97 mean score increase vs. Content Study: 1.42 mean score 
increase), despite receiving no intervention. 


The REMA was chosen for use in this study because it is the most appropriate standard measure available; 
currently, no standard assessments are designed to measure change in learning over such a short period of time, 
nor do any assessments cover the range of content addressed in the PEG+CAT materials. Therefore these findings 
suggest that the PEG+CAT resources show promise for supporting children’s mathematical learning and merit 
further study, with more rigorous research designs and more sensitive instrumentation. 


Child learning outcomes for PEG+CAT Item Sets (PCIS) 


For a small number of items on the PCIS, children’s post-test performance improved compared with their pre-test 
performance. Although gains were statistically significant for only one item (in which children were asked to identify 
a cylinder), the gains on a number of items are noteworthy. Below we provide more details about these items and 
offer some conjectures to help interpret these impacts on children’s mathematical skills. 


Children’s performance on an item that asked them to identify a cylinder, improved 
significantly between the pretest and the post-test. 


To assess their ability to recognize the 3-D shapes to which they were introduced in the PEG+CAT episodes, one 
set of items in the PCIS required the assessor to show the child images of characters from the PEG+CAT episode 
The Golden Pyramid Problem, and ask the child to name the shape the character is holding. One item in this set 
involved the assessor pointing to an image of Cat holding a cylinder (See Figure 5), and asking the child to identify 
the shape. Although no child was able to name the shape in the pretest, 17 of 58 children were able to correctly 
name the shape at posttest, a gain that was found to be statistically significant. 


In this instance, a number of features—both in The Golden Pyramid Figure 5. 
Problem as well as in the design of the assessment item—are 
likely to have reinforced children’s recognition of the cylinder. The 
cylinder (specifically, a “magic” cylinder) is central to the plot of the 
episode—the shape appears on the screen for an extended period 
of time; on numerous occasions, characters point to the shape 
and name it as a cylinder; on one occasion, one of the characters 
(Ramone) offers a definition of a cylinder (“a tube with circles on 
its ends”) and on another occasion, Peg compares a cylinder to a 
cube (“a cube has squares on the top and bottom, a cylinder has 
circles”). The assessment item used a screen capture to represent 
the cylinder in exactly the same way as children encounter it in the 
episode, held by Cat. 


The Golden Pyramid Problem 
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Although the episode and the corresponding suite of items in the assessment also included two other character-shape 
pairs (Toad holding a cube and Peg holding a rhombus), there were fewer features in the episode supporting children’s 
learning of these shapes in the same manner as for the cylinder. Children’s correct responses increased from pre- to 
post-test for the cube and the rhombus as well, but more modestly, from zero at pretest to 4 and 5 respectively. 


For anumber of other items, there were noteworthy gains in children’s performance 
between pre- and post-test. However, these gains were not statistically significant 
after adjusting for multiple comparisons. 


For five items—four focused on 3-D shapes and one focused on patterns—more children were able to provide 
the correct response during the posttest than during the pretest. Table 6 below highlights these items (please 
see the Appendix | for comprehensive tables showing the trends in children’s performance for all items). While 
not statistically significant, these gains are notable, as they suggest the potential of the PEG+CAT materials to 
enhance children’s understanding of 3-D shapes and patterns. 


Also notable is the fact that children’s gains are concentrated in the same session, in a suite of items that follow 
The Golden Pyramid Problem episode, which targets shapes. It is likely that some of the design features of the 
episode might have advanced children’s mathematical skill. In this episode, the shapes on which we measured 
learning gains—namely, the cylinder, cube, and pyramid— appear on the screen for a considerable length of time, 
and are defined (e.g., a pyramid has triangles on its sides), sometimes in terms of familiar referents (e.g., a cube 
looks like a box), and repeatedly identified (by the characters pointing to the shape while naming it). 


Finally, the greatest pre-post gains were observed in the items for which the post-tests were conducted during 
session 5, after children had had two opportunities to learn 3-D shape recognition from the PEG+CAT study 
experience—once in session 4, when they viewed The Sparkling Sphere Problem and played the Big Gig Cylinder 
game, and again in session 5, when they viewed The Golden Pyramid Problem and played the Magical Shape 
Hunt game. As such, it is likely that children may have benefited from two sessions worth of exposure to the 
mathematical skill of 3-D shape recognition. 
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Table 6. PCIS Items for Which Noteworthy Pre-Post Gains Were Observed 


PEG+CAT 


Episode/ 
Game 


Target 
Skill 


Description of item 


Using a screenshot from the 
PEG+CAT Game App, The Big Gig, 


Big Gig Pattern the assessor slides their fingers 
Patterns Completion across a 1, 3, 1,3 pattern, then 
asks the child to tell them what 
number comes next. 
The assessor places the following 
The Golden 3D shapes in a basket: sphere, 

; 3D Shape : : 
Pyramid FeeaeHiton cube, cylinder, and pyramid, then 
Problem 6 asks the child to point to or pick 

up the cube. 
The assessor shows the child 
images of various 2D and 3D 
THe aoe 3D Shape images from the PEG+CAT video, 
Pyramid he F 
rable recognition The Golden Pyramid Problem, and 
asks the child to point to the 
cylinder. 
The assessor places the following 
The Golden 3D shapes in a basket: sphere, 

: 3D Shape ; . 
Pyramid recouriien cube, cylinder, and pyramid, then 
Problem 6 asks the child to point to or pick 

up the cylinder. 

The assessor shows the child 

an image of Mermaid holding 
me eoldee 3D Shape pyramids from the PEG+CAT 
Pyramid ; ; 
Broblenm recognition video, The Golden Pyramid 

Problem, then asks the child what 

shape Mermaid is holding. 
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Pretest 


% 
Correct 


35.59% 


38.98% 


38.98% 


54.24% 


3.39% 


21 


23 


23 


32 


Posttest 


% 
Correct 


54.24% 


55.93% 


54.24% 


67.80% 


15.25% 


32 


33 


32 


40 


Change Pre 
to Post 


% 
Correct 


18.64% 


16.95% 


15.25% 


13.56% 


11.86% 


11 
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Parents’ views of and experiences with PEG+CAT materials 


Parents’ reflections of the PEG+CAT materials, including their potential to support children’s mathematical 
learning, are based on analyses of the Parent Checklist and Reflection Tool, completed by parents. Here we 
summarize parents’ views of the PEG+CAT videos and games; we also report on parents’ observations of their 
child’s engagement with these resources during and after the study sessions. 


Nearly all parents reported strong, positive impressions of PEG+CAT resources. 
Parents’ opinions of the videos were consistently high, though their views of the 
games varied. 


The vast majority of parents expressed appreciation of the PEG+CAT videos. Across sessions, the majority of 
parents (95%) agreed that the PEG+CAT videos were “good for young children”. Most parents indicated that 
children enjoyed watching all PEG+CAT episodes included in the study (91%), and that they understood the content 
presented in the video and what the characters were discussing (89.8%). There were no significant differences in 
parent reports by video episode. 


Likewise, most parents (95%) were of the opinion that the PEG+CAT interactive games were good for young 
children, and that children enjoyed engaging with the games during the study. There were few occasions (14% 
of sessions) when parents were concerned that their child was bored with the game. While parents had generally 
positive views of the games, opinions varied from game to game. In particular, parents expressed stronger positive 
opinions about Magical Shape Hunt and Chicken Dance compared to the games in the Big Gig? App , especially 
with respect to teaching problem solving (as Table 7 illustrates). 


Table 7. Parent Views of Games Addressing Problem Solving 


Percentage of parents who thought the 


PEG? CAT Game game addressed problem solving 
Magical Shape Hunt 94.7% (N = 54) 
Chicken Dance 92.7% (N = 51) 
Big Gig Magic Cylinder 74.5% (N = 41) 
Big Gig Patterns 74.5% (N = 38) 
Big Gig Count by Ones 72.2% (N = 39) 


The Magical Shape Hunt was especially popular among parents for its focus on problem solving (94.7%) and 
vocabulary (84.2%), and almost all parents (98.2%) reported that their child enjoyed playing this game. 


Overall (41%) the parents reported in Checklist responses that their children did not understand the purpose, 
rules, and procedures of at least one game, and that they needed help from an adult to start and continue playing 
the games. Parents expressed the most concerns that children found the Big Gig games confusing and difficult 


3 A new and improved version of the Big Gig app was released after the study concluded. 
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to play, specifically Big Gig Count by Ones (35.8% of parents) and Big Gig Patterns (36% of parents); in contrast, 
only a few parents (10.7%) expressed the same concern about Magical Shape Hunt. 


Parents reported that after PEG+CAT study sessions, they noticed changes in children’s 
behavior at home. 


In response to questions on the Parent Checklist and Reflection instrument, about half of parents (n=31, 54%) 
reported changes in their children’s behavior at home after study sessions. These parents reported hearing their 
children talk about songs, shapes, counting, vocabulary, questions from interactive games, patterns, numbers 
and colors. One parent noted that her son “likes to make patterns with his toys like the video game” while another 
parent reported that her daughter had become really interested in “counting, dancing, asking questions, and 
writing” after “she saw the PEG+CAT videos and games”. 


In more than half of participating families (64%; n=31), parents reported that children asked to watch PEG+CAT 
videos or play PEG+CAT games at home. While most families had the necessary equipment to view the videos 
at home (i.e., a television), almost half the parents (41%; n=24) reported they had no way of accessing PEG+CAT 
resources outside the study. For example, one parent commented, “No he tenido la oportunidad pero si me 
gustarla. (I haven’t had the opportunity but | would like to.)”, while another parent said, “No porque no tengo aseso 
a esos juegos. (No, because | don’t have access to these games.)”, and a third parent noted that her child wanted 
“to know when are we going to get PEG+CAT. | want to know how to get them.” 


Overall, parents viewed the PEG+CAT resources as having considerable potential to 
support children’s mathematical learning. 


In Checklist responses, nearly all parents remarked on the educational aims of the videos and games—that is, parents 
held the view that the PEG+CAT materials were trying to teach mathematics, problem solving, and new vocabulary. 


As the table below (Table 8) indicates, the majority of parents agreed that the PEG+CAT videos and games included 
in this study were addressing mathematics learning. In Checklist responses parents indicated that the mathematics 
topics covered by the videos and games were appropriate for preschool children. A couple of parents reported 
they liked how the mathematics topics were presented and taught in the videos. One said, “I think the video and 
game was good for my daughter. What caught my attention was patterns. She really loves patterns. | feel that it 


Table 8. Parent Views About the Content Upon Which Videos and Games Focused 


% of video viewing sessions % of game play sessions 

Yoel d=\--J] area a lem ne) ol (om (\ Eva: 72) addressing this topic (N=292) 
Mathematics 93.2% 88.1% 
Problem solving 96% 82% 
Vocabulary 82.7% 60.5% 
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teaches the kids a lot about patterns and shapes.” Another reported, “The teaching of the different shapes and 
sizes is the main thing that caught my attention. | loved the fact music is incorporated into the lesson. It makes 
learning easier.” 


Half of the participating parents reported that they worked with their children on 
activities related to PEG+CAT at home. 


In Parent Checklist and Reflection responses, half (N=29) of parents indicated that they had interacted with their 
children at home on PEG+CAT related activities. One parent pointed out that she had “the opportunity to work 
with my son doing mathematics activities, vocabulary, and questions about games etc.” Parents’ comments about 
home-based activities emphasized the mathematics content represented in the episodes included in the study. 
Recognizing and identifying shapes was the common topic that parents reported working on (29%; N=16). For 
example, one parent noted that, because “the last session was about shapes, we played identifying shapes and 
tried to name them just like the last video,” while another parent reported that she and her child “talk about shapes 
and count numbers.” 


Most parents (90%) indicated that they would watch the video or play the game with their child at home. One 
parent observed that she “enjoyed the activities as much as [her daughter]. The video was very funny and math- 
friendly. | would love to sit with my daughter and have fun watching and playing.” Another parent remarked that 
she “would definitely watch it and play the game because it’s educational. It shows numbers, patterns, and it’s a 
nice learning activity.” Finally, a parent who is also a preschool teacher reported that the videos and games gave 
her a few ideas and informed her teaching. 


Children’s engagement with the PEG+CAT experience 


Analyses of researchers’ observations and parents’ reflections paint a picture of how participating children engaged 
with the PEG+CAT videos and interactive games during the five weeks of the study. Here we describe how children 
attended to the videos and games, variations in the patterns of children’s attention, the amount and nature of 
adult support children required in order to engage with the media, and the extent to which children were able to 
summarize key elements of the videos and games. 


Overall, children were positively engaged while watching PEG+CAT videos and playing 
PEG+CAT games. 


Indicators of children’s engagement in the media included physical behaviors, such as sitting still and attending to the 
screen while watching videos or playing games, as well as verbal expressions of interest, such as singing, humming, 
or counting along with Peg, “interacting” with the characters by responding to or repeating their comments, asking 
questions about the videos and games, or requesting to watch the video or play the game again. 


Children paid focused attention during video viewings —for the vast majority of video viewing sessions (N=298), researchers 
noted that children sat still (91.2%) and paid attention to the events unfolding on the screen (94.9%). In some instances, 
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researchers also observed children clapping, snapping, or tapping their fingers; dancing, swaying, or nodding along in time 
to the music; pointing at the screen, leaning forward, or moving the laptop screen closer. One child was observed laughing, 
smiling, dancing in his seat, and singing along with PEG+CAT (to the 2, 4, 6, 8 song) while watching. 


Aside from physical behaviors, children also expressed their interest verbally. While watching The Beethoven Problem 
episode, for example, one child was observed laughing at the jokes, counting the sausages, and singing along with the 
characters in addition to leaning forward, with elbows on the table, watching intently. Across all video viewings (N=298), 
salient patterns of verbal engagement included laughing at jokes (86% of sessions); commenting on or asking questions 
about the videos (34.3%); singing along with Peg (20.5% of sessions); talking over the video or responding to the on- 
screen characters, by counting with Peg, and repeating things Peg said (e.g., shape names or mathematics problems), 
shouting out observations, etc. (16.2% of sessions). Although children appeared positively engaged during video viewing, 
requests to view the video again were rare, occurring only in 1.7% of the sessions. 


Researchers noticed that children exhibited a pattern of distracted or disengaged behavior in approximately one- 
fourth (26%) of all video-viewing sessions in the study. There was no evidence that distraction or disengagement 
was more common during particular videos. Indicators of disengagement included children looking around the 
room or out of the window rather than at the screen, putting their heads down, closing their eyes, fidgeting, hiding 
under the table, running around the room, spinning around in their chairs, asking to stop watching or to color with 
their siblings, and clicking out of the video screen. Other types of distraction were external or circumstantial (e.g., 
a sibling in the room or a loud noises in nearby rooms). 


Children were often engaged during PEG+CAT game play. For the majority of sessions (N =298), children remained 
seated (92.3%) and attended to the screen (94.3%) while playing. Researchers noted that children were focused on 
“winning” the game by completing it successfully; during game play sessions, children expressed excitement by 
clapping, yelling “Yes!” or “I did it!”, looking at observer or caregiver for affirmation, or pumping their fists into the air. 
Other expressions of children’s engagement included laughing, smiling, singing, dancing, trying to “interact” with the 
screen or the characters, narrating gameplay aloud (e.g., counting, naming shapes, etc.), and asking to play again 
and/or playing again on their own. While playing the Chicken Dance game, for example, a researcher noticed that the 
child “wanted to keep playing. Even after he accidentally clicked off the screen and it disappeared, he wanted to keep 
playing a few minutes later. He nodded his head when Peg asked him to help the chickens. He was smiling while 
watching and listening. He was making “cheep, cheep” sounds with his mouth, mimicking the chickens. He played 
the pattern game all the way through 4 times, persisting as the patterns got more complex. By the third round he 
was up from his chair, doing the dances, flapping his arms like a chicken. He was dancing along with the background 
music.” Similarly, another researcher observing a child during a session of the Magical Shape Hunt noted that the 
child smiled, laughed at various points during gameplay, and wanted to play the game multiple times. 


While children were more consistently engrossed during video viewing, their 
engagement with interactive games varied. 


With interactive games, children appeared to find some games more engaging than others. According to researchers’ 
observations, children seemed more engaged—expressing their interest physically and verbally—while playing the 
Magical Shape Hunt game (96.6%) and the Chicken Dance game (95.1%), than happened with other games. While 
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playing the Magical Shape Hunt game, a child enthusiastically responding “yes!” when the game asked, “Can you 
help the mermaid find her jewels?”, and counted out loud along with Ramone at various points in the game. 


Children were less engaged during the Big Gig* app sessions than they were during Chicken Dance and Magical 
Shape Hunt sessions. Some children became disengaged when they grew frustrated with an aspect of gameplay. 
Fewer children were reported trying to interact with all three activities in the Big Gig App (1.7%-13.3% compared 
to Chicken Dance and Magical Shape Hunt, both at 38.6%; p<0.001); fewer children wanted to play Big Gig: 
Counting by Ones (21.7%) and Big Gig: Patterns (19%; p=0.001); and Big Gig: Patterns had the largest proportion 
of children asking to stop playing (34.5%). 


Children’s apparent lack of engagement with the Big Gig App: Patterns might have been due to the issues they 
encountered while playing the game. For example, researchers noted that some children did not understand 
the purpose of the game or the need to press the numbers at the bottom of the screen, and pressed numbers 
randomly; in some instances, children were unable to play the games independently, and they required help and 
redirection from researchers. (See below for parents’ reflections of the Big Gig App, and their perceptions of the 
challenges encountered by the children while playing the games in this suite.) 


Most children were able to identify and talk about the different elements of the 
videos, but fewer were able to describe the problem and the solution at the center 
of each video episode. 


At the conclusion of video viewing, researchers asked children to share reflections on what they had noticed, most 
children were able to distinguish and discuss different elements of the episode. In 86.7% of sessions, children 
provided a verbal response, referring to the storyline, characters, mathematics concepts, or actions of specific 
characters. Children’s responses and their ability to recall plot, character, and other details related the episode may 
be seen as evidence of engagement (see Table 9 below for examples of children’s responses). 


Table 9. Children’s Reflections on the Video, Immediately After Viewing 


PEG+CAT Episode Children’s responses to the question, “Tell me about the video. What 


did you notice?” 


The little girl and her cat were singing with the bears and solving problems 


The Three Bears Problem and singing. 


The pig liked the song. A lot of people came to the 5th floor. The cat made 


Be Bectnoven Pravicin a new song, found a pattern and he laughed. 


The Sparkling Sphere Problem The dragon was sitting on the sphere and then on the oval. 


The Sparkling Sphere Problem | saw a rectangle. Rectangle prism. | saw the circle. 


Was about the king stole the triangles and the dragon was going to tickle them 
The Golden Pyramid Problem and then cat got a balloon and then Peg said, a cylinder is like this and cat was 
moving the tree and Peg caught the cylinder. That was my favorite part. 


4 The study used the original published version of the Big Gig. A newer version of the app became available after completion 
of the Content Study. 
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When asked about the problem (“What was the problem that Peg had?”) and the solution (“How did Peg and Cat 
solve the problem?”) at the center of each episode, children varied in how they responded (Table 10). 


Table 10. Children’s Reflections on the Problem and the Solution at the Center of 
Each Video 


Ke l=Talavicet-TaColameymaal-e)cede)(-100) Identification of the solution 


No / Incomprehensible 
response 


31.9% of sessions 37.9% of sessions 


Some response 67.8% of sessions 61.1% of sessions 


As Table YY indicates, a smaller majority of children were able to provide a response to questions about the 
problem/solution featured in each video. Describing The Golden Pyramid Problem, for example, one child noted, 
“The only problem | noticed was they couldn’t find the pyramids... They found them in the king’s castle in his 
hat”, while another child pointed out that the problem in The Dinosaur Problem episode was that “they (Peg and 
Cat) don’t know how to jump over big muddy puddle”, which they solved “by doing the pattern”. Other children 
provided details related to the mathematics and plot of the episode. For example, one child pointed out that in 
The Sparkling Sphere Problem, “they (Peg and Cat) had to save the wizard Ramone” by using “a sphere, [and a] 
rhombus”: “They counted from 2 to 20 and Ramone popped out of the bubble and he float in the bubble ...and he 
float[ed] in shapes.” 


Children were able to engage with the games independently more than half the 
time, but some form of support—typically from a researcher—was necessary for the 
other sessions. 


There were a number of reasons why children required assistance to start or continue with game play (46% of 
sessions). In some instances, children did not understand specific features of the game or the audio directions, or 
were not clear about what they were required to do. Most notably, this was an issue with the three games of the Big 
Gig App, where some children (approximately 30%) found it difficult to see a highlighted white line and understand 
that they were supposed to match the number that was in this field. Observing one child playing this game, for 
example, a researcher noted, “The child didn’t understand that the vertical, white highlighted area showed the 
number Cat wanted her to select. There were a handful of instances when she instead chose the next number in 
the pattern, getting the answer incorrect.” (Again, the version of the app available for this study has been replaced 
by an updated version that solved this and other user interface issues.) 


When children required some assistance, researchers modeled gameplay to help children who were unsure of how 
to begin a game; pointed out game elements they did not understand, such as the white highlighted line in the Big 
Gig App; showed children how to use aspects of the device, such as a computer mouse, camera, or track pad; and 
reminded children throughout gameplay about a certain game mechanics with which they were having difficulty (e.g., 
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clicking the mouse, waving hand over the camera). On a few occasions, parents provided support, usually in terms 
of gameplay and technology use. For example, one mother helped her child by instructing her in Spanish which 
numbers to press when playing the Big Gig App: Counting by Ones activity, while another mother helped her child 
use the track pad to play the Magical Shape Hunt game when the child kept trying to touch the screen. 


In some instances, children had difficulty with the equipment required to engage with the media. Approximately 
39% of children had trouble using a mouse or track pad correctly. Also, some children found the Magical Shape 
Hunt challenging—likely due to its novel use of the camera built into the Chromebook, requiring children’s hand 
movements to be within the camera’s range of view—something that is not necessarily intuitive to a preschooler 
who is unfamiliar with such equipment. 


There were a small number of technical difficulties (9.8% of sessions) during gameplay (e.g., slow Internet, 
disconnected wireless mouse, etc.) that required researchers to intervene. These predominantly occurred during 
online gameplay with Chicken Dance (20%) and Magical Shape Hunt (25.4%) and were related to a lapse in the 
wireless Internet connection. 
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Discussion 


This CPB-PBS Ready To Learn Content Study suggests that public media resources, and the transmedia strategy 
that has been the focus of Ready to Learn these past four years, have the potential to provide children with 
measurable benefits in the skill domains they seek to target. Assessment data, including statistically significant 
gains on one item and positive, non-significant gains on three items as noted above, indicate that that direct 
engagement with PEG+CAT transmedia resources shows promise for enhancing children’s mathematics skills 
(identification and recognition of 3-D shapes). These results are consistent with prior research studies using a 
similar design (Rockman et al., 2010). 


A large majority of study parents found PEG+CAT resources appealing, and characterized them as educational 
resources that can support mathematics learning for their preschoolers. Most parents (95%) reported that 
PEG+CAT videos and interactive games were good for young children, and there were few occasions (14% of 
sessions) when parents were concerned that their child was bored with a game. Finally, most children were able to 
engage with PEG+CAT video resources independently, and a majority were able and willing to use the PEG+CAT 
interactive games in a similar fashion. 


We recommend that these findings be interpreted with consideration of several 
additional observations about children’s performance on the assessments and 
behavior when using PEG+CAT resources. 


First, research concerning children’s development of mathematics concepts has documented particular learning 
trajectories, or the typical pathways of children’s emergent mathematical understanding. In particular, learning 
about patterns is more challenging, and comes developmentally later than learning about shapes, suggesting that 
these two skills should be separated and sequenced. In these frameworks, understanding the abstract idea of 
“patterns” develops gradually over early childhood (Clarke, Cheeseman & Clarke, 2006; Starkey, Klein & Wakely, 
2004), whereas a beginning understanding of shapes and the ability to distinguish between different shapes can 
be observed in very young children (Sarama & Clements, 2009). Further, geometric concepts, including shapes, 
have been described as the foundation for other mathematical thinking (Bronowski, 1947; Clements & Battista, 
1992; Clements & Sarama, 2007), suggesting strong shape knowledge and acquisition needs to be in place before 
more complex concepts, such as patterns, can be understood. As such, our findings may represent a natural 
developmental trajectory of young children’s mathematical learning. 
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Consistency and repetition are important for supporting children’s learning. 


The contextual nature of the presentation of concepts has been noted as key to children’s cognitive development 
(Vygotsky, 1987). Research shows children’s learning from media can deepen through successive interactions 
with specific concepts and skills (Crawley, Anderson, Wilder, Williams, & Santomero, 1999; Stevens and Penuel, 
2010). Children in the study did not have the opportunity to repeat experiences they found engaging and enjoyable 
as they may have done in a naturalistic setting. At the same time, children were repeatedly exposed to the same 
shapes across episodes and games, and the context for such exposure was the same—all episodes and games 
took place in the same setting. Further, the shapes were all defined multiple times with various examples, and as 
Sarama and Clements argue (2009), “shape words and names help organize and direct attention to the relevant 
features of objects” (209). Thus, the link between the naming of the shape and the presentation of the shape likely 
enhanced children’s learning. 


The notable exception to this pattern in our findings was children’s inability to identify a sphere, despite repeated 
exposure and verbal associations with objects. One possible explanation for this is that in the 2-D representation 
of videos and games, was not able to represent a 3-D sphere without making it look like a 2-D circle, such that 
children may have confused the two shapes. 


The way patterns were represented, explained, and reinforced may have been insufficiently clear and consistent. 
While children were provided with multiple opportunities to engage with patterns (i.e., watching a video and playing 
an app game), the available representations of the concept of patterns and specific pattern content may been less 
supportive of learning. Settings varied across episodes and between the episodes and games, and “pattern” 
was only defined once in the Beethoven episode and very briefly in the Big Gig Patterns game. Given that the 
development of children’s understanding of patterns is more gradual than it is for shapes (see above), it may be 
that this level of defining did not provide enough scaffolding for children to completely understand the conceptual 
idea of a pattern (Vygotsky, 1978). Further, the patterns children were exposed to in the episodes (AAB, AAAB) 
were more advanced than the traditional AB pattern, and also different from episode to episode. Whereas the 
same shapes were consistently presented, this lack of consistency suggests this presentation of patterns may 
have been too taxing on young children’s cognitive load (Sweller, Ayres, & Kalyuga, 2011). Similarly, in the Chicken 
Dance game, there is no mention of the word “pattern,” such that the game is not contextualized as a pattern 
but as helping the chickens complete their dance. Without a verbal cue or connection to the idea of “patterns,” 
children were not primed to think of the game as a pattern activity. Further, children received no feedback on their 
pattern completion, except the response from Peg: “That wasn’t the dance step that was supposed to come next, 
so, um try again?”, suggesting there was not sufficient scaffolding in place to help children refine their thinking as 
they attempted to provide the correct answer (Vygotsky, 1978). 
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Limitations and Constraints 


The sample of children for this exploratory, descriptive study was small, consisting of families living in urban 
neighborhoods, where there is a higher concentration of low-income households and children who speak 
a language other than English at home. While this sample is appropriate to the goals of the PEG+CAT study 
and broadly representative of the audience served by the Ready To Learn Initiative, the sample is not nationally 
representative of the population of preschool children and their families. Moreover, because the sample consisted 
of families who volunteered to participate in this study, selection bias (i.e., the participation of families where 
parents are favorably disposed toward media-rich forms of learning) cannot be ruled out. Given these limitations, 
the findings apply specifically to the children and families who participated in this study and do not support 
inferences about children in general. 


The study experience was unmediated in that children interacted with resources independently, and without 
cooperation or instruction from parents, teachers, or researchers (unless guidance was requested to deal with 
technical or usability issues that arose). The resources that children engaged with were carefully selected for their 
focus on patterns and shapes and the presence of particular educational strategies. They were also presented in 
a sequence that researchers felt would provide the strongest possible benefit. 


Study conditions may have influenced children’s behavior. Children and parents who participated in the study 
found themselves in a new and very unfamiliar environment: the offices of education researchers. This environment 
and the interactions with researchers who politely asked children to participate in study activities may have 
influenced children’s behavior. This effect may have been reinforced by the fact that families typically worked 
with, and received considerable individualized attention from, the same researcher over the course of the five- 
week experience, building rapport and a cordial relationship that made the experience more fun, but also may 
have created expectations on the part of the families about what was expected and how they should conduct 
themselves. Thus, children’s behavior and parents’ reflections may differ from what they would have been in more 
natural environments in which they interact with digital media. 


A customized approach to assessment was required. First, measures of the early mathematics skills targeted by 
PEG+CAT remain in short supply. Second, the scope and depth of the intervention itself were very limited, requiring 
an assessment that was similarly focused. Specifically, there are no standardized preschool mathematics measures 
with valid subtests for the particular skills (counting, patterns, shapes, etc.) targeted by the study experience. 
Assessments are generally designed to track learning as it emerges over longer periods of time and multiple learning 
experiences. The REMA was selected for this study because it was judged to be the best possible standard measure 
available, and has been shown to be valid and reliable with children from lower-income households. However, it 
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measures a broader set of skills than PEG+CAT resources covered, does not have sub-skill scales, and is designed 
to detect changes in learning over longer periods of time than those this study sought to examine. Similarly, the 
PCIS assessment was developed according to the highest research standards, but as with any curriculum-aligned 
assessment, it has limited psychometric data compared to established, standard measures. 


Data provided by parents through surveys was not triangulated with independent data collection activities such as 
observations in family homes. While researchers have no reason to mistrust the integrity of parent responses, but 
cannot exclude the possibility that parents may have sought to provide responses they felt researchers would prefer. 


The single-condition design means that all claims this research can support are more suggestive than definitive. 
Without a like comparison group of children who did not engage in study activities, it is not possible to exclude 
the possibility that differences in children’s performance resulted from chance experiences that took place outside 
the study. 
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Future Research 


This study’s findings point the way for research on transmedia and early learning, including our own Ready To 
Learn PEG+CAT Home Study (2014/2015). 


Early learning mathematics assessments 


The creation and validation of assessments that include subscales for individual skills (like patters and shapes) 
continues to be an important priority for the field. To date, no assessment instruments have been created that do 
this, or have been made valid and reliable for children living in under-resourced communities or for whom English 
is a second language. Similarly, there is a need for micro-genetic assessments that can capture emergent and 
partial knowledge, as opposed to the more typical approach to assessment that assumes a much longer period of 
engagement and rehearsal of new knowledge between pre- and post-test. 


The study transmedia experiences in the naturalistic settings 


While broad patterns of domestic media use and technology engagement have been documented through recent 
survey research (Rideout, 2014) much less is known about how contemporary families, siblings, and young children 
on their own engage with particular resources, or the degree to which children and parents may benefit from 
such engagements. New research, that takes place outside the learning laboratory conditions used in this study, 
could contribute new and important knowledge to the field by adding to the existing research on children’s home 
television experiences (Crawley et al, 1999; Anderson et al, 2000). 


The study of emerging learning technologies that are 
being broadly adopted 


Many studies have documented how children and families watch television. New research is needed to describe 
how children engage with touch-screen technologies like tablets and smart phones. In particular, new research 
on the time, manner, and social arrangements for such engagement can help developers to better understand 
the placement and function of these media in the context of home and school learning. In turn, this knowledge 
could inform public media’s efforts to build stronger home-school connections to better support early learning and 
development among children who may struggle with the skills needed to experience success in school settings. 
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A) PEG+CAT Content Study 


Appendix A: Flip Book 


RTL Content Study Assessment 


Welcome the child and establish rapport 


Today we are going to play a fun game together. Do you want to play with me? 


Session 2 Assessment 
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AA 


Session 2: Item 1 


Manipulatives: 
¢ 6 short sausages 


* 6 long sausages 


Ramone has some sausages; some are short and some are long. Peg wants him to 
make a pattern with the sausages. 


Help Ramone make a pattern here (point to space in front of the child) with these 
sausages (place the sausage images on the table in front of the child in a random order). 


Note: Any pattern produced is correct. 


PEG+CAT Content Study 


Session 2: Item 2. 


Manipulatives: 
¢ 6 short feathers 
¢ 6 long feathers 


Peg made a pattern with feathers: short, short, short, long, short, short, short, Jong 
(point to feathers on the flipbook as you read the pattern). 


Make the same pattern here (point to space in front of the child) using these feathers 
(place the feather images on the table in front of the child in a random order). 


3, ODDDSOOD, 
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Session 2: Item 3. 


No manipulatives 


Peg made a pattern using these numbers (slide finger from child’s left to right across the 
1, 3, 1, 3 pattern in the flipbook). 


What number comes next (slide fingers from left to right across the grey numbers)? 


Note: child may point to the number “1” within the pattern, within the greyed out numbers, 
or verbalize for correct answer. 
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Session 2: Item 4. 


Manipulatives 
¢ 6 short sausages 
* 6 long sausages 


Ramone made a pattern with sausages: short, short, short, long, short, short, short, 
long (point to sausages in the flipbook as you read the pattern). 


Make the same pattern here (point to space in front of the child) using these sausages 
(place the sausage images on the table in front of the child in a random order). 


Session 2: Item 5. 


Manipulatives: 
¢ 4 Lemons 
e 4Limes 


I'm making a pattern (from the child’s left to right, place a lemon, lime, lemon, lime pattern 
on the table while you say yellow, green, yellow, green). 


Now you keep going (place remaining limes in front of child). 


If child only puts one fruit, say: Keep going. 


Note: child only needs to add 1 unit for correct answer. 
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Session 2: Item 6. 


No manipulatives 


I'm going to make a pattern using my hands on the table. Let's look and listen. (Using 
both hands on the table slowly demonstrate: tap, tap, tap, clap; tap, tap, tap, clap; while at 
the same time saying the pattern: tap, tap, tap, clap; tap, tap, tap, clap. 


Make the same pattern using your hands. 


If child only completes tap, tap, tap, clap say: Keep going. 


Note: child can verbalize the pattern without the motion for correct answer 


Session 2: Item 7. 


Manipulatives: 
¢ 8 short guitars 
¢ 2 long guitars 


I'm going to try and make a short, short, short, Jong pattern using these guitars and | 
want you to help me make sure | make it right (from the child’s left to right, place guitars 
on the table between you and the child: short, short, short, long, short, short, short, short). 


| think | made a mistake...can you help me fix it (place the remaining guitars in front of 
the child)? Show me what | should change to make it right. 
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Session 2: Item 8. 


Manipulatives: 
¢ 6 number “1” cards (2 for pattern 4 for child) 
¢ 8 number “4” cards (4 for pattern 4 for child) 
¢ 4 of each: "2", "3" and "5" card 


| have these cards with numbers on them (lay out all of the number cards on the table 
between you and the child). |'m going to make a pattern using some of them (make the 
following ABB pattern from the child’s left to right so they can see: 1, 4, 4, 1, 4, 4). 


Make the same kind of pattern here (point to space below pattern) using these 
numbers (Point to the remaining number cards). 


Session 3 Assessment 
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Session 3: Item 1. 


Manipulatives: 
¢ 5 small dinosaur footprints 
¢ 5 large dinosaur footprints 


Peg and the dinosaurs like to make patterns in the mud with their footprints (point to 
Peg and Dinosaurs in the flipbook). 


Here are some dinosaur footprints (place footprints in random assortment in front of 
child); some are big and some are small. 


Help Peg and the dinosaurs make a pattern here (point to space in front of the child) 
with the footprints. 
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Session 3: Item 2. 


Manipulatives: 
¢ 4 purple blocks 
¢ 4 orange blocks 


Let's make a pattern (from child’s left to right, place: purple block, orange block, purple 
block, orange block). 


Show me what comes next (point to the space right next to the last block while you 
provide the blocks). 


If child only puts one block, say: Keep going. 


KKAK KA 
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Session 3: Item 3. 


Manipulatives: 
e Play dough 
¢ Shape stamps: star and triangle 


Look, I have these shape stamps. When we stamp in the play dough, we want to 
make soft stamps. Watch how | make soft stamps (model how to make a soft stamp by 
gently pressing the stamp into the play dough). 


Peg made a pattern (from the child’s left to right, slide your fingers through the pattern in 
the flipbook). 


Make the same pattern on the play dough (place rolled out strip of play dough in front of 
child) using these shape stamps (place stamps randomly in front of child). 


Note: shapes can be in any direction (e.g. upside down) for correct answer. 


Session 3: Item 4. 


No manipulatives 


I'm going to make a pattern using my hands. Watch me: (demonstrate with your hands 
fist, fist, open palm onto table; fist, fist, open palm onto table). 


Now it's your turn. Make the same pattern using your hands. 
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Session 3: Item 5. 


Manipulatives: 
¢ 4 orange squares 
¢ 4 purple triangles 
¢ 1 red diamond, 1 green circle, 1 blue upside down triangle 


Let's make a pattern like Peg (from the child’s left to right place: we. A. 


). 


, 


Which shape comes next? (Point to the space right next to the last shape while you 
provide the rest of the shapes in the order below). 


— 
= 
a 
aa 
a 
— 


1ut 
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Session 3: Item 6. 


Manipulatives: 
¢ 1 Dinosaur 


Peg and the dinosaurs like to make patterns in the mud with their footprints (point to 
Peg and Dinosaurs in the flipbook). The pattern they made is: (from child’s left to right, 
use dinosaur to demonstrate action and say step, step, bounce; step, step, bounce. 


Using this dinosaur (give the child the dinosaur), show me the pattern they made. 


Note: If the pattern is not visibly clear, say: oops, | missed that, please show me again? 
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Session 3: Item 7. 


Manipulatives: 
¢ 5 small dinosaur footprints 
¢ 5 large dinosaur footprints 


Peg and the dinosaurs made a pattern in the mud with their footprints (from child’s left 
to right, slide your fingers through the pattern in the flipbook). 


Here are some dinosaur footprints (place footprints in random assortment in front of 
child); some are big and some are small. 


Make the same pattern here (point to the space in front of the child) using the footprints. 


Session 3: Item 8. 


Manipulatives: 
¢ 3 orange squares 
¢ 4 purple triangles 
¢ 1 red diamond, 1 green circle, 1 blue upside down triangle 


I'm going to make a pattern like Peg (from child’s left to right place: teal A ea 
PA aan AN 


| forgot to put one shape. Which shape goes here? (Point to blank space while you 
provide the rest of the shapes in the order below). 
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Session 4 Assessment 
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Session 4: Item 1. 
No manipulatives 


Look at these shapes (point to top of page in the flipbook). Point to the shape 
that has squares on the top and bottom. 


PEG+CAT Content Study 


Session 4: Item 2. 
No manipulatives 


The mermaid needs help finding one of her jewels (run finger through shapes). 


Point to the jewel that is a solid shape that looks like a box with six square sides. 


Report to the Ready To Learn Initiative 


ag 


60 


Session 4: Item 3. 
No manipulatives 


What shape (point to blue cube in the flipbook) is the toad holding? 
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Session 4: Item 4. 
No manipulatives 


The mermaid needs help finding another jewel (run finger through shapes). 


Point to the jewel that looks like a tube with circles on the ends. 
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Session 4: Item 5. 
Manipulatives: 
¢ Basket of 3D shapes: Cube, sphere, cylinder, pyramid 


Here is a basket of shapes (provide child basket of shapes). 


Find the cylinder. 
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Session 4: Item 6. 
No manipulatives 


What shape (point to pyramid on the child’s left in the flipbook) is the mermaid holding? 


Report to the Ready To Learn Initiative 63 


Session 4: Item 7. 
Manipulatives: 
¢ Basket of 3D shapes: cube, sphere, cylinder, pyramid 


Here is a basket of shapes (provide child basket of shapes). 


Find the cube. 


64 PEG+CAT Content Study 


Session 4: Item 8. 
No manipulatives 


Look at these shapes (run finger through the shapes in the flipbook). 


Point to the sphere. 
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Session 4: Item 9. 
No manipulatives 


Look at these shapes (run finger through the shapes in the flipbook). 


Point to the cylinder. 
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Session 5 Assessment 
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Session 5: Items 1a — 1d 
No manipulatives 


Let's play a game. Look at these shapes (run finger through shapes in the flipbook). \'ll 
tell you what shape I am looking for and you can help me find it by pointing to it. 


1a. I'm looking for a flat shape that has four sides of equal length. Please point to it. 


1b. Now I'm looking for a shape that is round like a ball. Show me where that one is. 


1c. Now I'm looking for a shape with six flat sides that are all the same size. Where is 
that one? 


1d. Can you find a solid shape that has a rectangle on every side? Show me. 


For scoring: Each question (1a — 1d) is scored separately as correct or incorrect 
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Session 5: Items 2a — 2d 


Manipulatives: 
¢ 3D shapes: sphere, cone, rectangular prism, cylinder 


I'm going to show you some shapes and | need you to help me figure out what each 
of them is (place a sphere, a cone, a rectangular prism and a cylinder on the table). 


2a. Which one is the sphere? 
2b. Which is the cone? 
2c. Which is the rectangular prism? 


2d. Which is the cylinder? 


For scoring: Each question (2a — 2d) is scored separately as correct or incorrect 
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Session 5: Items 3a — 3b 
No manipulatives 


Look at all of these shapes (run finger through shapes in flipbook). Ramone needs your 
help finding specific shapes. I’ll point to the shape, and you say its name. Ready? 


3a. (point to the rectangular prism in the flipbook) 


3b. (point to the pyramid in the flipbook) 


For scoring: Each question (3a — 3b) is scored separately as correct or incorrect 
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Session 5: Items 4a 
No manipulatives 


Note: this Prompt is for the next 3 items — 4a — 4c 


I'm going to show you pictures of Peg and her friends and | want you to tell me what 
shape each of them is near. 


4a. Here is the mermaid (show the child a picture of the mermaid looking at the sphere in 
the flipbook). 


What shape is she looking at? 


Flip to next page 
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Session 5: Items 4b 


4b. Here is the toad (show the child a picture of the toad holding a cube in the flipbook). 


What shape is he holding? 


Flip to next page 


PEG+CAT Content Study 


Session 5: Items 4c 


4c. And here is Peg (show the child a picture of Peg holding a rhombus in the 
flipbook). 


What shape is she holding? 
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Session 5: Items 5 
No manipulatives 


Pig is standing near some shapes (point to picture of Pig with pyramids in the flipbook). 


Tell me, what shape is this? (point to the BLUE pyramid) 
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Session 5: Items 6 
No manipulatives 


Cat is holding a shape (point to picture of Cat holding a cylinder in the flipbook). 


Tell me, what shape is this? (point to the cylinder) 
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Appendix B: Observation Protocol 


Observation date: Observer: 


Child ID: Child Name: 


Study Session #: Peg + Cat Video Episode: 


Video Observation: 


Start time: LJLJ: O10 End time: L]LJ: OU 


1. What behaviors did you observe while the child was watching the video? [Mark all that apply.] 
Child was sitting still 


Child was moving around (e.g., standing up, moving around the table, 
etc.) 


Child was attending to the screen 


Child was turning away from the screen 


Child was laughing at jokes/funny moments in the video 


Child was attempting to “interact” with the characters by answering 
questions they asked or talking to them 


Child was singing along with characters 


Child was asking questions or making comments about the video 


Child asked to watch the video again 


Child appeared reluctant to watch the video 


Child asked to stop watching 


Other (Please describe): 


2. Based on your observations, do you think the child was engaged while watching the video? 
[Mark one]. 
Yes. Please explain. C] 


No. Please explain. Cl 


3. Ifchild was distracted or appeared disengaged, what do you think was the cause? [Mark all 
that apply.] 
Technical problem (e.g. technology malfunction like video freezing/ Cl 
not playing) 
Child wanted attention of parent/family more than wanting to watch 
the show 
Parent/family interrupted or engaged with child 


Other (Please describe): 
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4. Ifthe child made any comments or asked any questions during the video, please record them 
(and any parent/researcher responses) here. 


Follow-up prompts for child: 
Ask the child the following questions at the conclusion of the video. Please note their responses in 
the text boxes. 


1. Tell me about the video. What did you notice? 


2. What was the problem that Peg had? 


3. How did Peg and Cat solve the problem? 
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Observation date: Observer: 


Child ID: Child Name 


Study Session # Peg + Cat Game 


Game Observation: 


Start time: LJLJ: O10 Endtime: LILI? LO 


1. What behaviors did you observe in the child while playing the game? [Mark all that apply.] 
a. Child was sitting still fal 


b. Child was moving around (e.g., standing up, moving around the table, 
etc.) 


Child was attending to the screen 


Child was turning away from the screen 


Child was attempting to “interact” with the characters by answering 
questions or talking 


Child wanted to play the game more than once 


Child clicked randomly while playing 


Child was able to play the game without guidance or support 


Child was asking questions/ making comments about the game 


Child was moving around and turning away from the screen 


Child appeared reluctant to play the game 


Child asked to stop playing 


. Other (Please describe): 


Based on your observations, do you think the child was engaged while playing the game? 
[Mark one]. 
Yes. Please explain. Cc] 


No. Please explain. Cc] 


If child was distracted or appeared not to be engaged, what was the cause? 
a. Technical problem (e.g. technology malfunction like video freezing/ not 
playing) 
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Child wanted attention of parent/family more than wanting to watch the 
show 
Parent/family interrupted or engaged with child 


Other (Please describe): 


Did the child need support from researcher to begin playing? 
Yes. Please describe. 


Did any technical problems occur during game play? 
Yes. Please describe. 


Follow-up prompts for child: 
Ask the child the following questions at the conclusion of the video. Please note their responses in 
the text boxes. 


1. Tell me about the game. What did you notice? 


2. What did you think of the game? [Probe for whether children liked or disliked the game, 


why, whether they had a part they liked best, and so on]. 
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Appendix C: Parent Interview 


Interview Topics for Content Study 
When the opportunity arises, please ask parents any or all of the following questions. 


Name of Interviewer: 
Name of Parent: 
Child ID: 


We are really interested in learning more about how families use media at home and 
sharing what we learn with the producers who make videos and video games for 
children. Do you mind if | ask you some questions? Most of the questions are about the 
kinds of television shows or videos your child might watch or the video games your child 
might play. Does your child watch TV or videos or play video games at home? [/f the 
answer is no, the interview can end here.] 


1. Ok, tell mea little bit about the television shows your child watches. 


How about video games or computer games—could you tell me a little bit 
about the games your child plays? 


Are any of the shows CHILD watches or games s/he plays about learning 
math? /f yes: Tell me about those shows/games. What is CHILD learning? 


Are there things you and CHILD do at home that are about math? [/f parent 
doesn’t come up with anything, suggest counting together. | 


Are there other things you think CHILD is learning when s/he watches TV or 
plays video games? Of the shows CHILD watches and the games s/he plays, 
which do you think are the most educational? [Probe for specific show/game 
connections to learning—e.g., what does s/he learn from Sesame Street?] 


2. Weare really interested in learning more about how children use media at home— 
for example, do kids mostly watch TV or play video games on their own? Or is it 
sometimes with other kids (like brothers or sisters) or adults, like parents or 
babysitters? [Probe for when CHILD uses media, how often, with whom/frequency— 
we want an idea of how much is on their own, how much is with parents, how much 
is with siblings, etc.] 


What do you usually do during the time your child is watching a show or 
using technology? 


What kinds of help does CHILD need when using technology? [Probe on: Does 
CHILD ever have technical problems, like needing help getting to the right 
webpage or restarting the computer? Who provides assistance to CHILD 
when required?] 
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Do you ever have the chance to watch television or videos or play games 
with CHILD? Tell me about a time when you watched TV or played video 
games with CHILD. [Probe on a story — get some rich examples about a time 
when the parent and child watched television/ videos or played games 
together.] 


When you are watching TV or videos or playing games with CHILD, do you get 
the chance to ask CHILD questions? [/f yes: What kinds of questions?] 


Does CHILD ask you questions while s/he is watching TV or playing games 
with technology? What kinds of questions? 


What do you like about watching TV or videos or playing video games 
together with CHILD? 


Thank you for answering these questions! 
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Appendix D: Parent Survey (English-language version) 
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INTRODUCTION 


Thank you for helping us to learn more about how families with young children use technology. The 
information we are asking about will help researchers and program developers to make television shows 
and games that are fun and can help young children learn. Participating in this survey is voluntary. Please 
tell us about your use of technology and your child’s use of technology when he or she is at home, and 
answer some background questions about your family. If you have more than one child, please answer 
the questions thinking about the child who is participating in this study. 


Child ID: 


TECHNOLOGY AT HOME 


1. Please look over the list of devices in the left hand column. Do you have any in your household? 
(Mark all that apply.) 


| have this device in 
my household 


a. Television set 

b. Cable or satellite TV 

c. Paid video subscription (such as Hulu, Netflix, Amazon Prime) 

d. DVD or VHS player 

e. Laptop or desktop computer 

f. Smart phone (you can send email, watch videos, or access the Internet on it) 
g. Tablet (like an iPad, Galaxy Tab, Nexus 7, Microsoft Surface, or Kindle Fire) 
h. Basic e-reader (like a Nook or Kindle) 

i. Digital educational toys (like a LeapPad or VTech tablet) 

j. iPod Touch or other type of video iPod 

k. Video game player that hooks up to your TV (like an Xbox, PlayStation, or Wii) 


BO ee ee ee ee ee 


|. Handheld video game player (like a GameBoy, Nintendo DS or PSP) 


O 


m. Other (Please specify): 
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2. Please tell us what kind of Internet access you have at home (if any). (Mark only one). 
a. | don’t have Internet access at home 
b. Dial-up telephone access (through a modem) 
c. High-speed (such as broadband, cable modem or DSL) 
d. | have Internet access at home, but only on a cell phone 
e. | have Internet access at home, but | don’t know what kind 


f. | don’t know if | have Internet access at home 


NOUR Chilis USE OF IECEINOIOGY 


3. How often does your child do each of the following activities at home? 


1to2times 1to2times 3to4times Every 


eres a month a week a week day 


a. Watch TV 

b. Watch DVDs or videotapes 

c. Watch online videos 

d. Read or look at e-books (on an e-reader or other device) 


e. Play video games on a console or handheld video game 
player (such as Xbox, PlayStation, or Nintendo DS) 


f. Play games on a computer (laptop or desktop) 


g. Play games on mobile devices (such as cell phone, 
iPod, or iPad) 


h. Use apps other than games on mobile devices 


4. How often does your child do each of the following activities? 


1to2times 1to2times 3to4times Every 
a month a week a week day 


a. Watch educational television shows or DVDs im LC] ia im 


Never 


b. Watch educational videos online on a computer or 
mobile device L] fl L] [a] 


c. Play educational games on a video game player, 
computer, or mobile device U U U U 
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5. During the week (Monday-Friday), about how much time does your child spend using technology 
(watching TV, using the computer, playing video games, etc.) each day? (Mark only one). 


a. Less than half an hour a day 
b. Half an hour to 1 hour a day 
c. 1-2 hours a day 
d. 2-3 hours a day 


e. 3-4 hours a day 


f. More than 4 hours a day 


6. Do you monitor or limit your child’s technology use? (Mark only one). 


L]  a.No @ ski TO QUESTION 7. [] b.Yes 


6a. If you marked “yes” for Question 6 above, please tell us HOW you monitor or limit your 
child’s technology use. (Mark all that apply). 


a. | limit my child’s total time with technology. 

b. | schedule particular times of day my child can or cannot watch/play. 
c. | limit the content (particular shows/games) my child can watch/play. 
d. | limit which devices my child can use. 

e. | set limits by observing my child’s use of technology. 


f. | set limits based on child’s behavior (e.g., allow technology use for good behavior). 


g. Other: 


6b. If you marked “yes” for Question 6 above, please tell us WHY you monitor or limit your 
child’s technology use. (Mark ail that apply). 


a. I’m concerned about age-inappropriate content. 
b. | want my child to have time for other activities, like outdoor play or time with friends. 
c. I’m concerned about overuse of technology. 


d. I’m concerned about technology ‘addiction’. 


e. I’m concerned about physical activity levels, my child’s vision, or other health concerns 


SO eo 


f. Other: 
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MATH LEARNING AT HOME 


. Which of the following math skills, if any, do you help your child learn or practice at home or 
outside of school? (Mark all that apply.) 


. Counting (for example, counting out loud by saying “1, 2, 3” or counting objects such as 
cookies or the fingers on their hands) 


. Identifying written numbers (for example, the number “1” on the page of a book, or ona 
sign at the market) 


. Recognizing or drawing shapes (for example, recognizing or drawing circles, triangles, 
squares) 


. Recognizing or creating patterns (for example, laying out snack—cracker, grape, cracker, 
grape—or colored blocks—red, red, blue, red, red, blue—in a pattern) 


e. Addition (for example, knowing how many crackers you have when you add one to the pile) 
f. Subtraction (for example, knowing how many berries are left after eating some) 


g. Measurement (for example, measuring size with hands or rulers, measuring water with cups, or 
measuring ingredients for a recipe) 


O U aU eee oO eee 


h. Other (Please specify): 


8. Please indicate whether you agree or disagree with the following statements. 


Strongly 


Disagree Disagree Agree 


a. Young children (ages 3-5) learn math mainly at school, not at home LC] LC] LC] 


b. | can help my child learn math as well as other skills like reading 
and writing. 


c. | am confident in my ability to help my child learn and talk about 
mathematics. 


d. Young children (ages 3-5) generally like math and are interested in it. 


e. Math is too difficult for young children (ages 3-5) to understand. 


f. | like helping my child learn about math. 


g. Math can be taught anytime throughout the day. 


h. Part of my role is to teach math to my children. 
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NO CE] NOE) NID) 


9. Please indicate the highest level of 
education the child’s mother completed. 
(Mark only one). 


a. No formal schooling 

b. 8th grade or less 

c. 9th grade 

d. 10th grade 

e. 11th grade 

f. High School Diploma or GED 


g. Some college or technical school classes 
(no diploma) 


h. Associate’s Degree (AA, AS) or 
Technical Degree 


i. Bachelor's Degree (BA, BS) 
j. Graduate or Professional Degree 


k. Don’t know 


11. What languages do you speak at home? 


(Mark all that apply). 


a. English 

b. Spanish 

c. Chinese (Mandarin, Cantonese, or other) 
d. French 

e. Vietnamese 


f. Other (Please describe): 
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10. Please indicate the highest level of 


education the child’s father completed. 


(Mark only one). 
a. No formal schooling 
b. 8th grade or less 
c. 9th grade 
d. 10th grade 
e. 11th grade 
f. High School Diploma or GED 


g. Some college or technical school classes 
(no diploma) 


h. Associate’s Degree (AA, AS) or 
Technical Degree 


i. Bachelor’s Degree (BA, BS) 
j. Graduate or Professional Degree 


k. Don’t know 


12. Please indicate your race or ethnicity. 


(Mark all that apply). 
a. White (Non-Hispanic) 
b. Hispanic or Latino 
c. Black or African-American 
d. Asian 
e. Native Hawaiian or Other Pacific Islander 


f. American Indian or Alaska Native 


g. Other (Please specify): 
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13. What is your relationship to the child 14. Are you male or female? (Mark only one). 
listed on this survey? (Mark only one). SS 


a. Mother im a. Male LF b. Female 
b. Father 

c. Grandmother 

d. Grandfather 

e. Guardian 


f. Other (Please specify): 


THANK YOU VERY MUCH FOR TAKING THE TIME TO 
COMPLETE THIS SURVEY! 
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Estudio Peg + Cat 
mal@lUl- Kem) 0)(-ml-lerale)(ele|(om lan omnelaalii(e 


4 REapy TO" 


INUI;<@)DUL@Lel(O) 


Gracias por ayudarnos a aprender mas sobre como las familias con niftos pequefos utilizan la 
tecnologia. La informaci6n que solicitamos ayudara a los investigadores y desarrolladores de 
programas a producir programas de televisiOn y juegos que sean divertidos y que ayuden a los 
nifios pequefnos a aprender. La participaci6n en esta encuesta es voluntaria. Por favor, cuéntenos 
sobre su uso de la tecnologia y el uso de la tecnologia de su hijo cuando esta en casa, y conteste a 
algunas preguntas sobre su familia. Si usted tiene mas de un hijo, por favor conteste las preguntas 
pensando en el nifio que esta participando en este estudio. 


Identificacion del Nino(a): 


11 @IN(@) ROLE) Wa ENUROPWY\ 


1. Por favor, revise la lista de dispositivos en la columna de la izquierda. gTienes alguno de ellos en 
su hogar? (Por favor, marque todas las que correspondan.) 


Tengo éste en casa 


a. Television 

b. TV por cable o satélite 

c. Video de subscripci6n pagada (como Hulu, Netflix, Amazon Prime) 

d. Reproductor de DVD 0 VHS 

e. Computadora portatil o de escritorio 

f. Teléfono inteligente (puede enviar correos, ver videos, 0 acceder al internet) 
g. Tableta (como iPad, Galaxy Tab, Nexus 7, Microsoft Surface, o Kindle Fire) 
h. Lector electr6énico basico (como Nook o Kindle) 

i. Juguetes educativos digitales (como LeapPad o tableta VTech) 

j. iPod Touch u otro tipo de reproductor de video iPod 


k. Dispositivo de juegos conectado a TV (como Xbox, PlayStation, o Wii) 


OO ee ee ee ee 


|. Dispositivo de juegos portatil (como Game Boy, Nintendo DS o PSP) 


O 


m. Otro (Por favor explique): 
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2. gQué tipo de acceso a Internet tiene en su casa? (si tiene). (Varque solo una respuesta.) 


a. No tengo acceso a Internet en casa 

b. Acceso a internet por linea de teléfono (a través de un médem) 

c. Internet de alta velocidad (como banda ancha, mdédem de cable o DSL) 
d. Tengo acceso a Internet en casa, pero solo en teléfono celular 

e. Tengo acceso a Internet en casa, pero no sé de qué tipo 


f. No sé si tengo acceso a Internet en casa 


UO D) =e WW Ol NO) ROC] W2@)-aen ane 


3. gCon qué frecuencia su hijo(a) hace las siguientes actividades en casa? 


Unaodos Unaodos_ Tresocuatro Todos 
Nunca vecespor veces por veces por los 
mes semana semana dias 


a. Ver television 
b. Ver DVDs 0 cintas de video 


c. Ver videos online 


d. Leer o ver libros electrénicos (en un lector de libros 
electronicos u otro dispositivo) 


e. Jugar juegos de video en una consola 0 en dispositivo 
manual para juegos de video (como Xbox, PlayStation, 
or Nintendo DS) 


f. Jugar juegos en la computadora (portatil o de escritorio) 


g. Jugar juegos en dispositivos méviles (como teléfono 
celular, iPod, o iPad) 


h. Usar aplicaciones distintas de juegos en dispositivos 
moviles 


4. ~Con que frecuencia su hijo hace cada una de las siguientes actividades? 


Tres 0 
cuatro Todos 
veces por los dias 
semana 


Una o dos Una o dos 
Nunca veces por veces por 
mes semana 


a. Ve programas educacionales en la television o DVDs im L] | L] L 


b. Ve videos educacionales en linea, en computadora o en 
dispositivo movil LI LJ O LJ (a 


c. Juega juegos educativos en un dispositivo de juegos de 
video, en computadora, o en un dispositivo movil U U U U U 
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5. Durante la semana (lunes a viernes), cuanto tiempo del dia pasa su hijo usando la tecnologia 
(viendo television, usando la computadora, jugando juegos de video, etc.)? (Marque solo una respuesta.) 


a. Menos de media hora al dia 
b. Media hora a 1 hora al dia 
c. 1-2 horas del dia 

d. 2-3 horas del dia 


e. 3-4 horas del dia 


f. Mas de 4 horas del dia 


6. gMonitorea o limita el uso de tecnologia de su hijo? (Marque solo una respuesta.) 


[|] a.No sr0F) Si responde NO, por favor vaya a la pregunta 7. [ ]  b.Yes 


6a. Si marco “si “ en la pregunta 6, por favor diganos COMO controla o limita el uso de la 
tecnologia de su hijo. Por favor, marque todas las respuestas que correspondan. 
(Marque todas las respuestas que correspondan.) 


a. Limito el tiempo total que mi hijo usa la tecnologia. 

b. Programo momentos concretos del dia en que mi hijo puede o no ver/jugar. 

c. Limito el contenido (solo algunos programas/juegos) que mi hijo puede ver/jugar. 
d. Limito los dispositivos que mi hijo puede usar. 


e. Pongo limites observando el uso que mi hijo hace de la tecnologia. 


f. Pongo limites basados en el comportamiento del nifio (por ejemplo, permitir el uso de 
tecnologia por buen comportamiento). 


g. Otro (Por favor explique): 


. Si marco “si “ en la pregunta 6, por favor diganos POR QUE controla o limita el uso de la 
tecnologia de su hijo. (Marque todas las respuestas que correspondan.) 
a. Estoy preocupado por contenido inapropiado para la edad. 


b. Quiero que mi hijo tenga tiempo para otras actividades, como jugar al aire libre o 
pasar tiempo con sus amigos. 


c. Estoy preocupado por el uso excesivo de la tecnologia. 
d. Estoy preocupado por la "adiccién" a la tecnologia. 


e. Estoy preocupado por los niveles de actividad fisica, la vision, u otros problemas de 
salud de mi hijo. 


f. Otro (Por favor explique): 
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APRENDIZAJE DE MATEMATICAS EN EL HOGAR 


7. De las siguientes habilidades matematicas, con cuales le ayuda a su hijo a aprender o practicar 
en casa o fuera de la escuela? (Marque todas las respuestas que correspondan.) 


a. Contando (por ejemplo, contar en voz alta diciendo: "1, 2, 3" o contar objetos como 
galletas o los dedos de la mano) 


b. Identificando numeros escritos (por ejemplo, reconocer el numero “1” en la pagina de un 
libro, o en un letrero de la tienda) 


c. Reconociendo formas o dibujos de formas (por ejemplo, reconociendo o dibujando 
circulos, triangulos, cuadrados) 


d. Reconociendo o creando patrones (por ejemplo, reconociendo un aperitivo—galleta, uva, 
galleta, uva—o bloques de colores—rojo, rojo, azul, rojo, rojo, azul—en un patr6n) 


e. Sumando (por ejemplo, sabiendo cuantas galletas habra si se agrega una mas a una pila) 
f. Restando (por ejemplo, sabiendo cuantas frutillas quedan después de comerse algunas) 


g. Midiendo (por ejemplo, midiendo tamafio con las manos 0 reglas, midiendo agua con tazas, 0 
midiendo los ingredientes para una receta) 


O UO ope Oo pee 


h. Otro (Por favor explique): 


8. Por favor indique qué tanto esta usted de acuerdo o en desacuerdo con lo siguiente. 


Muy en En De Muy de 
desacuerdo desacuerdo acuerdo acuerdo 


a. Los nifos pequefios (edades 3-5) aprenden matematicas Og g gO Oo 
sobre todo en la escuela, no en casa. 


b. Yo puedo ayudar a mi hijo a aprender matematicas asi 
como otras habilidades como lectura y escritura. 


c. Tengo capacidad para ayudar a mi hijo a aprender y hablar 
de las matematicas. 


d. Alos nifios pequefios (edades 3-5) en general les gustan e 
interesan las matematicas. 


e. Las matematicas son muy dificiles para que nifios pequefhos 
(edades 3-5) las entiendan. 
f. Me gusta ayudar a mi hijo para que aprenda matematicas. 


g. Las matematicas pueden ensefarse en cualquier momento 
del dia. 


h. Parte de mi funcién es ensefar matematicas a mis hijos. 
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INTRO) IVC O)NING)=] UNA NI H@teB) = NIT =S 


9. Por favor indique el nivel mas alto de 
educacion que termino la madre del nino. 


10. Por favor indique el nivel mas alto de 
educacion que termin6o el padre del nino. 


(Marque solo una respuesta.) 


a. Sin educacién formal 
b. Grado 8° o menos 

c. Grado 9° 

d. Grado 10° 

e. Grado 11° 


f. Diploma de preparatoria (High School) 
o GED 


g. Algunas clases de la universidad o 
escuela técnica (sin diploma) 


h. Grado Técnico Asociado (AA, AS) o Grado 
Técnico 

i. Licenciatura (BA, BS) 

j. Postgrado o grado profesional 


k. Nolo sé 


ORO oO ee Oe ee 


11. ~Qué idioma(s) habla usted en casa? (Marque 
todas las respuestas que correspondan.) 


a. Inglés 

b. Espanol 

c. Chino (mandarin, cantonés u otro) 
d. Francés 

e. Vietnamita 


f. Otro (Por favor describa): 
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(Marque solo una respuesta.) 


a. Sin educacién formal 
b. Grado 8° o menos 

c. Grado 9° 

d. Grado 10° 

e. Grado 11° 


f. Diploma de preparatoria (High School) 
o GED 


g. Algunas clases de la universidad o 
escuela técnica (sin diploma) 


h. Grado Técnico Asociado (AA, AS) o Grado 
Técnico 


i. Licenciatura (BA, BS) 
j. Postgrado o grado profesional 


k. No lo sé 


12. Por favor indique su raza u origen étnico. 


ORO oO Bee Oe ee 


(Marque todas las respuestas que correspondan.) 


a. Blanco (No Hispano) 

b. Hispano o Latino 

c. Negro 0 afro-americano 

d. Asiatico 

e. Hawaiano o de las Islas del Pacifico 
f. Indio Americano o Nativo de Alaska 


g. Otro (Por favor especificar): 
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13. ~Cual es su relacion con el nifo que se 14. gEs usted hombre o mujer? (Mark only one). 
nombra en esta encuesta? (Marque solo as 


una respuesta.) 
L] a. Hombre im b. Mujer 


a. Madre 
b. Padre 
c. Abuela 
d. Abuelo 


e. Tutor 


f. Otro (Por favor describa): 


jGRACIAS POR PARTICIPAR EN ESTA ENCUESTA Y POR CONTRIBUIR CON 
OK@]0)] =x @X@) (OO! = (OND) =A RONEN NOD AWW n@)\ (@)ROlC1/\ 
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Appendix F: Checklist and Reflection (English-language version) 


Child ID: Child Name: Parent Name: 


Date: Session # 


Please observe while your child watches the video. Check one box for each of the questions below. 


Name of Video: Yes 


Z 
° 


a. Have you seen this video before? 


b. Do you think your child is enjoying this video? 


Do you think your child is bored with this video? 


c 
d. Do you think this video is trying to teach anything? 


Do you think this video is trying to teach math? 


Do you think this video is trying to teach children how to solve problems? 


Do you think this video is trying to teach children vocabulary or new 
words? 


Do you think your child understands what the characters are saying/ 
talking about? 


i. Does your child seem confused by this video? 


j._ Do you think this is a good video for young children? 
Other comments: 


Please observe while your child plays the game and check one box for each of the questions below. 


Z 
° 


Name of the Game: Yes 


a. Have you seen this game before? 


b. Do you think your child is enjoying this game? 


Do you think your child is bored with this game? 


c 
d. Do you think this game is trying to teach anything? 


Do you think this game is trying to teach math? 


Do you think this game is trying to teach children how to solve problems? 


Do you think this game is trying to teach children vocabulary or new 
words? 


h. Does your child seem confused by this game? 


i. Do you think your child needs help from an adult when playing this game? 


j.__Do you think this is a good game for young children? 
Other comments: 
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We are interested in learning more about your reactions to the video and game, so we can share that 
feedback with the developers. Please respond to the following questions, providing as much detail as 
possible. 


1. What did you think about the video and the game? Please describe what caught your attention as 
you were watching. 


Was there something in the video or game that you did not understand or did not like? Please 
explain. 


Do you think your child would like to watch this video or play this game? Please explain. 


Do you think you would watch this video or play this game with your child? Why or why not? 


Have you had the opportunity to work with your child at home on activities related to the Peg + 
Cat videos and games? Please describe. 


Has your child asked to watch Peg + Cat or play the Peg + Cat video games at home? 


Have you observed any change in your child’s behavior at home related to the Peg + Cat videos 
and games? For example, does your child act like Peg or Cat at home? 
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Appendix G: Checklist and Reflection (Spanish-language version) 


ID del nifio: Nombre del nifio: Nombre del Padre/Madre: 


Fecha: Sesién Nimero: 


Por favor observe mientras su hijo ve el video. Marque un solo cuadro para cada pregunta. 


Nombre del video: 


a. Ha visto usted este video antes? 


iCree usted que su hijo esta disfrutando este video? 


iCree usted que su hijo se aburre con este video? 


iCree usted que este video esta tratando de ensefiar algo? 


iCree que este video esta tratando de ensefiar matematicas? 


iCree usted que este video esta tratando de ensefiar a los nifios 
como resolver problemas? 


iCree usted que este video esta tratando de ensefnar a los nifos 
vocabulario o palabras nuevas? 


iCree usted que tu hijo entiende lo que los personajes del video 
estan diciendo o de lo que estan hablando? 


i. ¢Parece su nifo confundido con este video? 


OO; OF; OF OOOO OO 
OO) OF | OJOJOJO;O Os 


j. @Cree usted que este es un buen video para nifios pequefios? 
Otros comentarios: 


Por favor observe, mientras que su hijo juega el juego y marque un solo cuadro para cada pregunta. 


Nombre del juego: 


a. {Ha visto usted este juego antes? 


iCree usted que su hijo esta disfrutando este juego? 


iCree usted que tu hijo se aburre con este juego? 


d. ;Cree usted que este juego esta tratando de ensenar algo? 


iCree que este juego esta tratando de ensenhar matematicas? 


iCree que este juego esta tratando de ensefnar a los nifos como 
resolver problemas? 


O OOJOJO OO 
OO O}OVOJO OO 


iCree que este juego esta tratando de ensejfar a los nifios 
vocabulario o palabras nuevas? 
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~Parece su nifo confundido por este juego? 


iCree usted que su hijo necesita ayuda de un adulto cuando 
juega este juego? 


j. @Cree usted que este es un buen juego para nifios pequenios? 
Otros comentarios: 


Estamos interesados en entender mas sus reacciones ante el video y el juego, para poder retroalimentar 
con esa informacion a quienes los desarrollaron. Por favor responda a las siguientes preguntas, 
proporcionando tantos detalles como sea posible. 


1. gQué piensa sobre el video y el juego? Por favor describa lo que le Ilam6 la atencion, mientras estaba 
observando. 


2. jHubo algo en el video o juego que usted no entendid o que no le gust? Por favor, explique. 


3. Cree usted que a su hijo le gustaria ver este video o jugar a este juego? Por favor, explique. 


4. Cree usted que le gustaria ver este video o jugar a este juego con su hijo? ;Por qué si o por qué no? 


5. Ha tenido la oportunidad de trabajar con su hijo en casa en actividades relacionadas con los videos y 
juegos Peg + Cat? Por favor describa. 


6. ,Le ha pedido su hijo ver el video Peg + Cat 0 jugar los videojuegos de Peg + Cat en casa? 


7. jHa observado algtin cambio en el comportamiento de su hijo en casa, relacionado con los videos y 
juegos de Peg + Cat? Por ejemplo, ,Actia su hijo como Peg o Cat en su casa? 
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RTL Content Assessment Study: Session 2 - Sets 2 and 3 Pretest 


Child Name 
Child ID 


Assesor 


Date 


Prompt 


Correct Response 


Score 


Help Ramone make a pattern here with these sausages 


2 repeating units 


correct 


incorrect 


Make the same pattern here using these feathers 


short short short long x 2 


correct 


incorrect 


What number comes next? 


Make the same pattern here 
Now you keep going. 


1 verbally or 1 pointing 
short short short long x 2 


add lemon lime x1 


correct 
correct 


correct 


incorrect 
incorrect 


incorrect 


Make the same pattern using your hands. 


tap, tap, tap, clap x 2 


correct 


incorrect 


Show me what | should change to make it right. 


indicate last short should be long 


correct 


incorrect 


Make the same kind of pattern here using these numbers 


Help Peg and the dinosaurs make a pattern here with the 
footprints. 


Show me what comes next 
Make the same pattern on the playdough using these 
shape stamps 


any ABB x2 


2 repeating units 


add 1 unit (purple block, orange block) 


star, star, triangle x 2 


correct 


correct 


correct 


correct 


incorrect 


incorrect 


incorrect 


incorrect 


Now it's your turn. Make the same pattern using your 
hands. 


fist, fist, open palm x 2 


correct 


incorrect 


Which shape comes next? 


orange square 


correct 


incorrect 


Using this dinosaur show me the pattern they made. 


step, step, bounce x 2 


correct 


incorrect 


Make the same pattern here using the footprints. 


| forgot to put one shape. Which shape goes here? 
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small, small, large x 2 


orange square 


correct 


correct 


incorrect 


incorrect 


Appendix I: 


PCIS Item Results 


Table I1. Session 2 Results PCIS Items 


Correct 
XA of) aX] 


Description 


Pretest Posttest 


to Post 


Change Pre 


1 Pattern 
Creation 
2 Pattern 
Extension 
3 Pattern 
Duplication 
4 Pattern 
Duplication 
Pattern 
5 
Completion 
Pattern 
6 pNniicse 
uplication 
Pattern 
7 ee 
Duplication 
Pattern 
8 ; 
Completion 


The child is asked to create a 


pattern using dinosaur footprints 2 repeating 
from The PEG+CAT video, The units 
Dinosaur Problem. 

The assessor creates an ABAB ada iunit 


pattern with purple and orange 
blocks, then asks the child to extend 
the pattern using the blocks 


(purple block, 
orange block) 


The assessor shows the child an 
image of an AABAAB pattern of red 
stars and yellow triangles, then asks 
child to duplicate the pattern in play 
dough using 


star, star, 
triangle x 2 


The assessor demonstrates the 
pattern with their hands: fist, fist, 
open palm onto table; fist, fist, open 
palm onto table, then asks the child 
to duplicate the same pattern. 


fist, fist, open 
palm x 2 


Using the Chicken Dance images 
from The PEG+CAT Online Game, 
Chicken Dance, the assessor makes 
an ABAB pattern, then asks the 
child to pick from all the Chicken 
Dance images to complete the 
pattern. 


orange 
square 


Using a plastic toy dinosaur, the 
assessor demonstrates an AAB 
pattern (step, step, bounce), and 
then asks the child to duplicate the 
pattern using the dinosaur. 


step, step, 
bounce x 2 


The assessor shows child an image 
of the dinosaur footprint pattern 
step, step, bounce, then the 

child is asked to create the same 
pattern using dinosaur footprints 
from PEG+CAT video, The Dinosaur 
Problem. 


small, small, 
large x 2 


Using the Chicken Dance images 
from The PEG+CAT Online Game, 
Chicken Dance, assessor makes an 
ABAB _B pattern, then asks the child 
to pick from all the Chicken Dance 
images to insert the correct piece of 
the pattern. 


orange 
square 


58 


58 


58 


58 


58 


58 


58 


58 


%Correct n %Correct n %Correct 
29.31% 17 31.03% 18 1.72% 
84.48% 49 79.31% 46 -5.17% 
53.45% 31 56.90% 33 3.45% 
58.62% 34 62.07% 36 3.45% 
56.90% 33 63.79% 37 6.90% 
70.69% 41 79.31% 46 8.62% 
36.21% 21 32.76% 19 -3.45% 
65.52% 38 63.79% 37 -1.72% 
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Table I2. Session 3 Results PCIS Items 


Correct Change Pre 


Description Response Pretest Posttest fo\Post 


%Correct n %Correctt n %Correct n 


The child is asked to create a 


Pattern pattern using sausages from The 2 repeating ‘ 9 9 
1 Creation PEG+CAT video, The Beethoven units Be, Pewee ea nee ee, ee is 
Problem. 


Using images of feathers in lengths 
of short and long, the assessor 
makes an AAAB pattern, and then 
asks the child to duplicate the 
pattern using the feathers. 


short short 
short long x 2 


Pattern 
Duplication 


uw 
oO 


42.37% 25 47.46% 28 5.08% 3 


Using a screenshot from the 
PEG+CAT Game App, The Big Gig, 
3 Pattern the assessor slides their fingers 1 verbally or 
Completion across a 1,3, 1,3 pattern, then asks 1 pointing 
the child to tell them what number 
comes next. 


59 35.59% 21 54.24% 32 18.64% 11 


The assessor shows the child 
images of sausages arranged in 


an AAABAAAB pattern from The 
4 bed tion PEG*CAT video, The Beethoven a cle 58 44.83% 26 50.00% 29 5.17% 3 
UPHCAatlOn  b-oblem. The Child is then asked SHLOTE KONE 
to duplicate the pattern using the 
sausages provided. 
The assessor creates an ABAB 
pattern with plastic lemons and 
5 cecal limes, then asks the child to extend grant 58 72.41% 42 81.03% 47 862% 5 


the pattern using the lemons and 
limes provided. 


The assessor demonstrates the 
. Pattern pattern with their hands: tap, tap, tap, tap, tap, 
Duplication clap; tap, tap, clap, then asks the clap x 2 
child to duplicate the same pattern. 


58 36.21% 21 43.10% 25 6.90% 4 


The assessor tells child they are 
making an AAABAAAB pattern using 


images of guitars. The assessor indicate last 
Pattern them makes an AAAABAAAA short guitar é ‘ 4 
7 Recognition pattern on the table, and tells should be a ee ee ee 
the child to help them make sure long 


they did it right, and to help them 
change it to make it right. 


Using the numbered circles from 
the PEG+CAT Game App, The Big 
Gig, the assessor creates the 

8 Pattern pattern 1,4,4,1,4,4 (from the game), 

Duplication and then gives the child numbered 

circles 1-5 asking them to create 
the same kind of pattern using the 
numbers provided. 


any ABB x2 = 57) = 50.88% 29 61.40% 35 10.53% 6 
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Table I3. Session 4 Results PCIS Items 


Item 


3a-3b 
Prompt 


3a 


3b 


Target Skill 


Shape 
characteristics 


Shape 
characteristics 


Shape 
characteristics 


Shape 
characteristics 


3D Shape 
recognition 
3D Shape 
recognition 
3D Shape 
recognition 


3D Shape 
recognition 


3D Shape 
recognition 


3D Shape 
recognition 


Description 


The assessor shows the child 
images of various 2D and 3D 
images from the PEG+CAT video, 
The Sparkling Sphere, then using 
definitions from the video, asks 
the child to point to the shape 
that fits the definition 


I'm looking for a flat shape that 
has four sides of equal length. 


Now I'm looking for a shape that 
is round like a ball. 


Now I'm looking for a shape with 
six flat sides that are all the same 
size. 


Can you find a solid shape that 
has arectangle on every side? 


For the items 2a-2d, the assessor 
places the following 3D shapes 
on the table: sphere, cone, 
rectangular prism and cylinder, 
then asks the child to point to or 
pick up the shape. 


Which one is the sphere? 
Which is the cone? 
Which is the rectangular prism? 


Which the cylinder? 3D items 


The assessor shows the child 
images of various 2D and 3D 
shapes from the PEG+CAT video, 
The Sparkling Sphere, and asks 
the child to say the name of the 
shape. 


Rectangular prism 


Pyramid 


Correct 
XA oo) aX) 


Rhombus 


Sphere 


Cube 


Rectangular 
Prism 


Sphere 


Cone 


Rectangular 
Prism 


Cylinder 


Rectangular 
prism 


Pyramid 


57 


57 


57 


57 


57 


57 


57 


57 


57 


57 


Pretest 


% Correct 


19.30% 


91.23% 


21.05% 


29.82% 


31.58% 


71.93% 


63.16% 


54.39% 


0.00% 


8.77% 


n 


36 


31 


Posttest 
% Correct n 
22.81% 13 
91.23% 52 
21.05% 12 
22.81% 13 
29.82% 17 
80.70% 46 
68.42% 39 
43.86% 25 
5.26% 3 
10.53% 6 


Change Pre 
to Post 
% Correct on 
3.51% 2 
0.00% 0 
0.00% 0 
-7.02% -4 
-1.75% ~ -1 
8.77% 5 
5.26% 3 
-10.53% -6 
5.36% 3 
1.75% 1 
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Table 13. Session 4 Results PCIS Items (Continued) 


Correct Change Pre 


Item Target Skill Description Response N Pretest Posttest to Post 


%Correctt n %Correctt n %Correct n 


The assessor shows the child 
various images of characters 
from the PEG+CAT video, The 


tind Sparkling Sphere, and asks the 
P child to say the name of the 
shape the character is holding, or 
near. 
3D Shape The assessor points to Mermaid F i ‘ 
5 recognition looking at a sphere aphere ot ee" . een ! aids : 
3D Shape The assessor points to Toad F i fi 
4b recognition holding a cube Cube 57 0.00% OO 7.02% 4 7.02% 4 
2D Shape The assessor points to Peg 6 - i 
Ac recognition _ holding a rhombus Rhombus 57 0.00% OO 877% 5 877% 5 
3D Shape The assessor points to Pig near a F é - F 
5 recognition pyramid Pyramid 57 5.26% 3 10.53% 6 5.26% 3 
Be SnRRE THE See eee) Piso Cot Cylinder 57 0.00% 0 29.82% 17 29.82% 17 


recognition —_ holding a cylinder 
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Table 14. Session 5 Results PCIS Items 


Item Target Skill 


1 Shape 
characteristics 
2 Shape 
characteristics 
3 3D Shape 
recognition 
4 Shape 
characteristics 
5 3D Shape 
recognition 
6 3D Shape 
recognition 
7 3D Shape 
recognition 
8 3D Shape 
recognition 
9 3D Shape 
recognition 


Description 


The assessor shows the child images of 
acube and a pyramid from The PEG+CAT 
video, The Golden Pyramid Problem, then, 
using the definition from the video, the 
assessor asks the child to "point to the 
shape that has a square on the top and 
bottom". 


The assessor shows the child an image of 
the Mermaid with various 3D shapes from 
The PEG+CAT video, The Golden Pyramid 
Problem, then, using the definition from 
the video, the assessor asks the child to 
"point to the jewel that is a solid shape 
that looks like a box with six square sides". 


The assessor shows the child an image of 
Toad holding a cube from The PEG+CAT 
video, The Golden Pyramid Problem, 

then asks the child what shape Toad is 
holding. 


The assessor shows the child an image of 
the Mermaid with various 3D shapes from 
The PEG+CAT video, The Golden Pyramid 
Problem, then, using the definition from the 
video, the assessor asks the child to "point 
to the jewel that looks like a tube with 
circles on the ends". 


The assessor places the following 3D 
shapes in a basket: sphere, cube, cylinder, 
and pyramid, then asks the child to point to 
or pick up the cylinder. 


The assessor shows the child an image 
of Mermaid holding pyramids from 

The PEG+CAT video, The Golden Pyramid 
Problem, then asks the child what shape 
Mermaid is holding. 


The assessor places the following 3D 
shapes in a basket: sphere, cube, cylinder, 
and pyramid, then asks the child to point to 
or pick up the cube. 


The assessor shows the child images 

of various 2D and 3D images from the 
PEG+CAT video, The Golden Pyramid 
Problem, and asks the child to point to the 
sphere. 


The assessor shows the child images 

of various 2D and 3D images from the 
PEG+CAT video, The Golden Pyramid 
Problem, and asks the child to point to the 
cylinder. 


Correct 
Response 


Cube 


Cube 


Cube 


Cylinder 


Cylinder 


Pyramid 


Cube 


Sphere 


Cylinder 


N 


59 


59 


59 


59 


59 


59 


59 


59 


59 


Pretest 


% Correct n 


81.36% 


81.36% 


3.39% 


33.90% 


54.24% 


3.39% 


38.98% 


16.95% 


38.98% 


48 


48 


20 


32 


23 


10 


23 


Posttest 


% Correct n 


88.14% 


93.22% 


8.47% 


44.07% 


67.80% 


15.25% 


55.93% 


20.34% 


54.24% 


52 


55 


26 


40 


33 


32 


Change Pre 


to Post 


% Correct n 


6.78% 


11.86% 


5.08% 


10.17% 


13.56% 


11.86% 


16.95% 


3.39% 


15.25% 
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